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Notice to Limited English Proficient Persons 


Notice of Language Assistance: If you have difficulty understanding English, you may request language 
assistance services, free of charge, for this Department information by calling 1-800-USA-LEARN (1-800-872-5327) 


(TTY: 1-800-877-8339), or e-mail us at: Ed.Lanquage.Assistance@ed.gov. 


[SPANISH] 

Aviso a personas con dominio limitado del idioma inglés: Si usted tiene alguna dificultad en entender el idioma 
inglés, puede, sin costo alguno, solicitar asistencia lingUistica con respecto a esta informacién llamando al 1-800- 
USA-LEARN (1-800-872-5327) (TTY: 1-800-877-8339), o envie un mensaje de correo electrénico a: 


Ed.Language.Assistance@ed.gov. 


[CHINESE] 

PMRBREAARALHBA: MRETERB, RESARGAAHE, CIVUSRKRESAAMERMBAS MRA, 
RRGSERA RMA. SHtBa psa Beek. MREBSAMOSRSSRBMHAMAM, GMB 1- 
800-USA-LEARN (1-800-872-5327) (Hesai2 ALS : 1-800-877-8339), MER: 
Ed.Language.Assistance@ed.gov. 


[VIETNAMESE] 

Thong bao danh cho nhieng ngwoi co kha nang Anh ngiv han ché: Néu quy. vi gap khé khan trong viéc hiéu 
Anh ng@ thi quy vi co thé yéu cau cac dich vu hé tro ngén ng@ cho cac tin tc cua BO danh cho cong chung. Cac 
dich vu hé tro ngén ng® nay déu mién phi. Néu quy vi muén biét thém chi tiét vé cdc dich vu phién dich hay thong 
dich, xin vui long ggi sO 1-800-USA-LEARN (1-800-872-5327) (TTY: 1-800-877-8339), hoac e-mail: 


Ed.Language.Assistance@ed.gov. 


[KOREAN] 

S30 SNS Att 2D: SHS ONE GU AASO! WA SY, cs Ae Aled] SRO! CHAP AO Ale! 
AMbuIAS ea Sol aleteel OAS AN AA MBAS SSF WSSU SAOLE HS AYlAQ! Chol AAA St 
ABO Beotdl 3S, Ast S 1-800-USA-LEARN (1-800-872-5327) 4 ee A2t SoelS Ast S 1-800-877-8339 
f= ogdnaen Sa pu neay meirisie ine en OF ASotaAl7| HraliCt. 
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[TAGALOG] 

Paunawa sa mga Taong Limitado ang Kaalaman sa English: Kung nahihirapan kayong makaintindi ng English, 
maaari kayong humingi ng tulong ukol dito sa inpormasyon ng Kagawaran mula sa nagbibigay ng serbisyo na 
pagtulong kaugnay ng wika. Ang serbisyo na pagtulong kaugnay ng wika ay libre. Kung kailangan ninyo ng dagdag 
na impormasyon tungkol sa mga serbisyo kaugnay ng pagpapaliwanag o pagsasalin, mangyari lamang tumawag sa 
1-800-USA-LEARN (1-800-872-5327) (TTY: 1-800-877-8339), o mag-e-mail sa: Ed.Language.Assistance@ed.gov. 


[RUSSIAN] 

YBeOM.1eHHe 1A JIM ¢C O(pPaHH4eHHbIM 3HaHHeM aHTIMCKOLO A3bIKa: EciM BbI MCIIbITBIBaeTe 
TPYAHOCTH B IOHHMAaHHN aHTIMHCKOrO A3bIKa, BbI MOXKETE MOMPOCHTh, YTOOBI BAM IIpeAOcTaBHJIM MepeBoy 
uHdopMaunu, KoTopyro MunnctepcTBo Oobpa30BaHHa JOBOANT WO BceoOulero cBeAeHHA. ITOT MepeBor 
lipeqoctapiaetca OecrmaTHO. Ec BbI XOTUTe MOVANT Ooee MOApOOHy!o HHMopMawMto OO yclyrax ycTHOrO 
HW IIMCbMeHHOTO MepeBosa, 3BOHUTe 110 Tee@ouy 1-800-USA-LEARN (1-800-872-5327) (ciy2x0a Wa 
cnadocibimlallux: 1-800-877-8339), um oTMpaBbTe coobijeHHe Mo ampecy: Ed.Lanquage.Assistance@ed.gov. 


Please submit your comments and questions regarding this plan and report and any suggestions to improve future 
reports, including suggestions for additional links that will increase the usefulness of the report to the public, to 


APP_APRComments@ed.gov or: 


U.S. Department of Education 
Performance Improvement Officer 
400 Maryland Ave, SW 
Washington, DC 20202 


The following companies were contracted to assist in the preparation of the U.S. Department of Education 
FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan: 


For general layout and web design: ICF Macro 
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Foreword 


As required by the Government Performance and Results Act (GPRA) Modernization Act of 2010, each federal 
agency must report annually on its progress in meeting the goals and objectives established by its Strategic 
Plan. The United States Department of Education’s (the Department’s) Fiscal Year (FY) 2016 Annual 
Performance Report and FY 2018 Annual Performance Plan presents to Congress, the President, and the 
American people detailed information about progress in meeting the Department’s strategic goals and 
objectives and performance metrics. This report accompanies the administration’s budget request to Congress. 
The complete budget request for the Department will be available at 


http:/Awww.ed.gov/about/overview/focus/performance.html. 


This year, the Department is consolidating its FY 2016 Annual Performance Report and the FY 2018 Annual 
Performance Plan into one report on its U.S. Department of Education Strategic Plan for Fiscal Years 2014— 
2018. The data included in this report for the FY 2014-18 Strategic Plan metrics are the most current data 
available to the Department. The Department’s FY 2016 annual reporting includes these two documents: 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan 
[available May 2017] 


This report is produced in conjunction with the FY 2018 President's Budget Request and provides more detailed performance 
information and analysis of performance results. 


FY 2016 Agency Financial Report (AFR) [published November 14, 2016] 


The AFR is organized into three major sections: 


e The Management's Discussion and Analysis section provides executive-level information on the Department’s 
history, mission, organization, key activities, analysis of financial statements, systems, controls and legal 
compliance, accomplishments for the fiscal year, and management and performance challenges facing the 
Department. 


The Financial section provides a Message From the Chief Financial Officer, consolidated and combined financial 
statements, the Department’s notes to the financial statements, and the Report of the Independent Auditors. 


The Other Information section provides improper payments reporting details and other statutory reporting 
requirements. 


All annual reports will be available on the Department’s website at 
http://www2.ed.gov/about/reports/annual/index.html. 
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Our Mission 


The U.S. Department of Education’s mission is to promote student 
achievement and preparation for global competitiveness by fostering 
educational excellence and ensuring equal access. 


Mission and Organizational Structure 


Who We Are. In 1867, the federal government recognized that furthering education was a 
national priority and created a federal education agency to collect and report statistical data. 
The Department was established as a cabinet-level agency in 1980. Today, the Department 
supports programs in every area and level of education. 


Our Public Benefit. While recognizing the primary role of states and school districts in 
providing a high-quality education, the Department is committed to helping ensure that students 
throughout the nation develop skills to succeed in school, college, and the workforce. The 
Department supports efforts to help students succeed regardless of background or 
circumstance by encouraging challenging content, setting high expectations for all students, and 
monitoring academic progress. 


The Department's largest fiscal responsibility is a portfolio of student loans (see the Financial 
Highlights and Notes sections of the Agency Financial Report [AFR]). The second-largest fiscal 
responsibility is grants to states, mostly for elementary and secondary education, awarded 
based on both statutory formulas (see the chart on page 6 of the AFR) and competitions. The 
third largest fiscal responsibility is student aid to help pay for college through Pell Grants, Work 
Study, and other campus-based programs (see the Notes section of the AFR). 


The Department also supports research, collects education statistics, enforces civil rights 
statutes, and carries out competitive grant programs to promote innovation (see The 
Department’s Approach to Performance Management section). 
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Our Organization in Fiscal Year 2016 


MISSION AND ORGANIZATIONAL STRUCTURE 


This chart reflects the coordinating structure of the U.S. Department of Education in FY 2016. 


Office of the 
Deputy Secretary 


Performance 
Improvement Office 


Risk Management 
Service 


Office of Small and 
Disadvantaged Business 
Utilization 
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Office of the 


Secretary 
Chief of Staff 


Office of 
Inspector General 


Institute of 


Education Sciences 


Office of Communications 
and Outreach 


Office for 
Civil Rights 


Office of Legislation and 
Congressional Affairs 


yx 


Office of Planning, 
Evaluation and Policy 
Development 


“ 
Office of Elementary and 
Secondary Education 


Office of Innovation 
and Improvement 


Office of Special >) 
Education and 
Rehabilitative Services 


Office of English 
Language Acquisition 


International 
Affairs Office 


Office of the 
Under Secretary 


Federal Student Aid 


Office of Career, 
Technical, and 
Adult Education 


Office of Postsecondary 
Education 


WH Initiative on 
Educational Excellence for 
African Americans 


WH Initiative on Asian 
Americans and Pacific 
Islanders 


WH Initiative on 
Educational Excellence for 
Hispanics 


WH Initiative on 


Historically Black Colleges 
and Universites 


WH Initiative on 
American Indian and 
Alaskan Native Education 


Center for Faith-based 
and Neighborhood 
Partnerships 


About This Report 


The United States Department of Education’s (the Department’s) FY 2016 Annual Performance 
Report (APR) and FY 2018 Annual Performance Plan (APP) provide information relative to the 
FY 2014-18 Strategic Plan. 


About the Agency Financial Report 


The AFR is available on the Department’s website. The Management’s Discussion and Analysis 
section highlights information on the Department’s performance, financial statements, systems 
and controls, compliance with laws and regulations, and actions taken or planned to address 
select challenges. The Department’s intent is to provide users with access to helpful information 
about the Department and its financial and performance activities. To help continue to improve 
the content of the AFR, readers are encouraged to provide their feedback at 


AERComments@ed.gov. 
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PERFORMANCE RESULTS DETAILS 


Performance Results Details 


Performance Management Framework 


In accordance with the Government Performance and Results Act (GPRA) Modernization Act of 


2010, the Strategic Plan is the basis for the Department’s performance management framework. 
The Department uses quarterly performance reviews, targeted strategic initiatives, and outreach 
to leaders and stakeholders to assess progress and garner engagement toward achieving 
strategic goals and outcomes. 


The performance data presented in this report are based on the goals in the Department's FY 
2014—18 Strategic Plan (outlined below). The Department welcomes input from Congress, state 
and local partners, and other education stakeholders on its Strategic Plan and Agency Priority 
Goals (APGs). Questions or comments about the Department’s performance management 
framework and reporting should be e-mailed to PlO@ed.gov. 


FY 2014-18 Strategic Plan 


GOAL 5: ——_ 
ahaa U.S. Department 
Improvement of the asta 
U.S. Education GOAL 1: of Education 


Postsecondary Education, Capacity 


Career and Technical Education, ¢ 
and Adult Education . 


System 
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PERFORMANCE RESULTS DETAILS 


The FY 2014-18 Strategic Plan is comprised of six strategic goals, which serve as the 
foundation for establishing long-term priorities. Within each goal are strategic objectives that the 
Department undertakes to realize the goal. For each objective, the Department has established 
measures to gauge its progress. Additionally, in collaboration with the Office of Management 
and Budget (OMB), the Department established FY 2016-17 APGs and reported quarterly 
updates on these APGs on performance.gov. 


The Department monitors progress toward its strategic goals and its APGs using data-driven 
review and analysis. This focus promotes active management engagement across the 
Department. 


FY 2014-18 Strategic Plan* 


SSS 


Mission: To promote student achievement and preparation for global competitiveness 
by fostering educational excellence and ensuring equal access 


FY 2014-18 STRATEGIC PLAN 


FY 2016-17 
Agency Priority Goals 


iS) (=1(-\e | (om © e)- 13 


S) (r=) (Ye (om @] 0) [lees 


Objective 1.1: Access and Affordability. Close the opportunity gap 

by improving the affordability of and access to college and/or workforce 

z training, especially for underrepresented and/or underprepared 
Education, Career populations (e.g., low-income and first-generation students, English 

and Technical learners, individuals with disabilities, adults without high school diplomas, 
Education, and etc.). 

Adult Education. 


Goal 1: 
Postsecondary 


+ Increase college 
degree attainment in 


Objective 1.2: Quality. Foster institutional value to ensure that America 


Increase college 
access, affordability, 
quality, and completion 
by improving 
postsecondary 
education and lifelong 
learning opportunities 
for youths and adults. 


Goal 2: Elementary 
and Secondary 
Education. |mprove 
the elementary and 
secondary education 
system's ability to 
consistently deliver 
excellent instruction 
aligned with rigorous 
academic standards 
while providing 
effective support 
services to close 
achievement and 
opportunity gaps, and 
ensure all students 
graduate high school 
college- and career- 
ready. 


postsecondary education credentials represent effective preparation for 
students to succeed in the workforce and participate in civic life. 


Federal Student Aid 
Objective 1.3: Completion. Increase degree and certificate completion Transparency 
and job placement in high-need and high-skill areas, particularly among 


underrepresented and/or underprepared populations. 


Objective 1.4: Science, Technology, Engineering, and Mathematics 
Pathways. |ncrease STEM pathway opportunities that enable access to 
and completion of postsecondary programs. 


Objective 2.1: Standards and Assessments. Support implementation 
of internationally benchmarked college- and career-ready standards, with 
aligned, valid, and reliable assessments. 


Objective 2.2: Effective Teachers and Strong Leaders. |mprove the 

preparation, recruitment, retention, development, support, evaluation, 

recognition, and equitable distribution of effective teachers and leaders. + Support 
implementation of 
college- and career- 
ready standards and 
assessments 


Objective 2.3: School Climate and Community. Increase the success, 
safety, and health of students, particularly in high-need schools, and 
deepen family and community engagement. 


Objective 2.4: Turn Around Schools and Close Achievement Gaps. 
Accelerate achievement by supporting states and districts in turning 
around low-performing schools and closing achievement gaps, and 
developing models of next-generation high schools. 


Objective 2.5: STEM Teaching and Learning. \ncrease the number 
and quality of STEM teachers and increase opportunities for students to 
access rich STEM learning experiences. 


' This graphic reflects the Strategic Plan adopted by the Department in 2014. 
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Strategic Goals 


PERFORMANCE RESULTS DETAILS 


FY 2014-18 STRATEGIC PLAN 


FY 2016-17 


Strategic Objectives Agency Priority Goals 


Goal 3: Early 
Learning. |mprove 
the health, social- 
emotional, and 
cognitive outcomes for 
all children from birth 
through 3rd grade, 
so that all children, 
particularly those with 
high needs, are on 
track for graduating 
from high school 
college- and career- 
ready. 


Goal 4: Equity. 
Increase educational 
opportunities 

for underserved 
students and reduce 
discrimination so that 
all students are well- 
positioned to succeed. 


Goal 5: Continuous 
Improvement of 

the U.S. Education 
System. Enhance the 
education system’s 
ability to continuously 
improve through 
better and more 
widespread use of 
data, research and 
evaluation, evidence, 
transparency, 
innovation, and 
technology. 


Goal 6: U.S. 
Department of 
Education Capacity. 
Improve the 
organizational 
capacities of the 
Department to 
implement the 
Strategic Plan. 


Objective 3.1: Access to High-Quality Programs and Services. 
Increase access to high-quality early learning programs and 
comprehensive services, especially for children with high needs. 


Objective 3.2: Effective Workforce. |mprove the quality and 
effectiveness of the early learning workforce so that early childhood 
educators have the knowledge, skills, and abilities necessary to improve 
young children’s health, social-emotional, and cognitive outcomes. 


Objective 3.3: Measuring Progress, Outcomes, and Readiness. 


Improve the capacity of states and early learning programs to develop 
and implement comprehensive early learning assessment systems. 


* Increase enrollment 
in high-quality state 
preschool programs 


Objective 4.1: Equitable Educational Opportunities. |ncrease all 
students’ access to educational opportunities with a focus on closing 
achievement gaps and remove barriers that students face based on 
their race, ethnicity, or national origin; sex; sexual orientation; gender 
identity or expression; disability; English language ability; religion; 
socioeconomic status; or geographical location. 

Objective 4.2: Civil Rights Compliance. Ensure educational 
institutions’ awareness of and compliance with federal civil rights 
obligations and enhance the public's knowledge of their civil rights. 


+ Ensure equitable 
educational 
opportunities 


Objective 5.1: Data Systems and Transparency. Facilitate the 
development of interoperable longitudinal data systems for early learning 
through employment to enable data-driven, transparent decision-making 
by increasing access to timely, reliable, and high-value data. 


Objective 5.2: Privacy. Provide all education stakeholders, from early 
childhood to adult learning, with technical assistance and guidance to 
help them protect student privacy while effectively managing and using 
student information. 


Objective 5.3: Research, Evaluation, and Use of Evidence. \|nvest in 
research and evaluation that builds evidence for education improvement; 
communicate findings effectively; and drive the use of evidence in 
decision-making by internal and external stakeholders. 


Objective 5.4: Technology and Innovation. Accelerate the 
development and broad adoption of new, effective programs, processes, 
and strategies, including education technology. 


* Enable evidence- 
based decision 
making 


Objective 6.1: Effective Workforce. Continue to build a high-performing, 
skilled, diverse, and engaged workforce within the Department. 


Objective 6.2: Risk Management. |mprove the Department's program 
efficacy through comprehensive risk management, and grant and 
contract monitoring. 


Objective 6.3: Implementation and Support. Build Department 
capacity and systems to support states’ and other grantees’ 
implementation of reforms that result in improved outcomes, and keep 
the public informed of promising practices and new reform initiatives. 


Objective 6.4: Productivity and Performance Management. |mprove 
workforce productivity through information technology enhancements, 
telework expansion efforts, more effective process performance 
management systems, and state-of-the-art leadership and knowledge 
management practices. 
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PERFORMANCE RESULTS DETAILS 


Cross-Agency Priority (CAP) Goals 


Cross-Agency Priority (CAP) Goals are used to accelerate progress on a limited number of 
priority areas where implementation requires active collaboration between multiple agencies. 


The 15 CAP Goals include the following 7 mission-oriented and 8 management-focused goals. 


Mission CAP Goals Management CAP Goals 


e Cybersecurity e Customer Service 

e Service Members and Veterans e Smarter IT Delivery 
Mental Health e Category Management 

e Climate Change (Federal Actions) e Shared Services 

e Insider Threat and Security Clearance e Benchmark and Improve Mission- 
Reform Support Operations 

e Job-Creating Investment e Open Data 

e Infrastructure Permitting e Lab-To-Market 
Modernization e People and Culture 

e STEM Education 


The Department contributes to the following four CAP Goals: 


Cybersecurity Goal Statement: Improve awareness of security practices, vulnerabilities, and 
threats to the operating environment by limiting access to only authorized users and 
implementing technologies and processes that reduce the risk from malicious activity. 


Service Members and Veterans Mental Health Goal Statement: Improve mental health 
outcomes for Service Members, Veterans, and their Families. 


Science, Technology, Engineering, and Math (STEM) Education Goal Statement: Improve 
STEM Education by implementing the Federal STEM Education 5-Year Strategic Plan, 
announced in May 2013. 


Customer Service Goal Statement: Deliver world-class customer services to citizens by 
making it faster and easier for individuals and businesses to complete transactions and have a 
positive experience with government. 
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PERFORMANCE RESULTS DETAILS 


The Department’s Approach to Data Collection and Analysis 


In FY 2016, the Department continued to support programs to help the education system by 
facilitating the development of the infrastructure necessary to collect and disseminate high-value 
education information for the improvement of student outcomes. 


EDFacts. The EDFacts system enables the consolidation of separate school-, district-, and 
state-level data collections and reduces the reporting burden for states by eliminating redundant 
data requests. 


Statewide Longitudinal Data Systems. The Statewide Longitudinal Data Systems (SLDS) 
grant program, as authorized by the Educational Technical Assistance Act of 2002, is designed 


to aid state educational agencies (SEAs) in developing and implementing longitudinal data 
systems. Most SLDS funds are awarded as state grants, but a portion of the funds are used for 
activities to improve data quality, coordination, and use. Activities include the Education Data 
Technical Assistance program and the Privacy Technical Assistance Center. 


Data Strategy Team. The Department’s Data Strategy Team (DST) develops and promotes 
coordinated and consistent data strategies among the various offices whose responsibilities 
include data and content management within the Department. The mission of the DST is to 
coordinate the Department’s public-facing data initiatives by building cohesiveness in internal 
processes and data policies and by improving transparency in matters related to the 
Department’s collection of data. 


Civil Rights Data Collection (CRDC). The Department collects data on key education and civil 
rights issues in our nation’s public schools for use by the Office for Civil Rights (OCR) in its 
enforcement and monitoring efforts, by other Department offices, and by policymakers and 
researchers outside of the Department. 


Enhancing Education Systems and Supports. The Department strives to leverage its data, 
evaluation, performance, and financial systems to meet four important aspects of its mission: 


e Tocontribute to the Department’s ability to build customer relations by providing timely 
responses to customer inquiries. 


e To empower employees to make informed decisions by increasing their access to data. 

e To increase accountability through improved financial management. 

e To keep Department employees informed of the project status and ensure that all users 
receive proper training on the new system. 


Support for the Department’s Evaluations 


To determine the effectiveness of programs, policies, and strategies for improving education 
outcomes, funding is directed toward evaluations that will yield valid, reliable, and useful 
information for the field. For a list of evaluations completed in FY 2016 and of those planned 
through FY 2018, see appendix D. 
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Performance Plan Summary 
Looking Ahead and Addressing Challenges 


With new leadership, the Department of Education begins a fresh start for educational 
opportunity and innovation in America. For too long, educators have had to spend significant 
time filling out paperwork and seeking permission to enact commonsense reforms. In the future, 
this agency will be a mirror—reflecting and illuminating the tremendous work that educators do 
every day with the support of parents, advocates, current and former students, and community 
members. 


Our goal is to support greater state and local flexibility in elementary and secondary education, 
already underway with implementation of the Every Student Succeeds Act, while helping every 
student receive a high-quality education. Rather than merely enforcing compliance, the 
Department will empower educators. The Department also will embolden parents and students 
to make their own choices, whether traditional public, public charter, private, or other innovative 
options. Our guiding principle for assisting this work is simple: Let teachers teach, let students 
learn, and let parents decide the path their child’s education will take. 


Greater flexibility must take root in higher education as well. That means innovative new models 
for delivering an affordable, accessible, high-quality postsecondary education that holds all 
institutions accountable for their use of taxpayer funds. The Department will also seek to 
improve and, if necessary, repeal regulations that needlessly burden institutions of higher 
education and drive up costs and debt for students. This will allow colleges and universities to 
spend less time on paperwork and bureaucracy and more time on instruction and research. 


The Department has a key role to play in enforcing the law and ensuring every student—and 
especially the most vulnerable—has access to a high-quality education and receives the 
protections guaranteed them under the law. This important work will be strengthened by leaders 
in state capitals, district offices, and college administration buildings having the opportunity to do 
things differently to meet the needs of their students. Greater flexibility will mean more 
opportunities for students, but only if those new possibilities are explored and new ideas are 
encouraged. 


Therefore, over the next year, the Department will: 


e Identify internal processes and regulations that are out-of-date, are not reflective of best 
practices, are legally insufficient, or are too cumbersome for teachers, principals, and 
parents. In every case, why a regulation is necessary and reasonable will need to be 
demonstrated. 


e Proactively reach out to educators, parents, institutions, and state and local leaders to 
identify opportunities for additional flexibility and support. 


e Work with members of Congress from both parties to identify additional opportunities to 
reduce the red tape that encumbers or distracts educational institutions from their central 
mission of educating their students. 


e Ensure that billions of dollars in federal student aid will be spent effectively, producing 
positive outcomes for students and a solid return on investment for the taxpayer and 
society. 
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While decisions made by local government will be valued, there is no educational decision more 
local or more important than that made by a child’s parent or guardian. The Department will 
support parental choice—not as an endorsement of private, charter, magnet, virtual, or 
traditional public schools, but as an acknowledgment of parental wisdom and authority to make 
decisions important to their child’s future. High-quality data and transparency at the state, local, 
and school levels will further empower parents to make the best decisions for their children. 


Many have favored school turnaround efforts over parental choice. Under the School 
Improvement Grants (SIG)—$7 billion was spent in recent years alone and a comprehensive 
evaluation of the program found “no significant impacts” on any student outcome. 


If we can identify a school turnaround model that shows promise, we want to learn about it. If we 
find a solution that demonstrates consistent results, we want to support it. However, we cannot 
wait and hope for a miracle, while blocking efforts that can help millions of children immediately. 
That is why we must invest in what we know works for students even as we continue to innovate 
and build knowledge. 


We must also refocus our efforts—and our federal, state, and local partnerships—on helping 
Americans get ahead and stay ahead by finding a fulfilling and well-paying career. Career and 
technical education must be aligned with state standards and directed towards areas of greatest 
need where there is the greatest potential for a good job. Although the needs of employers must 
be considered to a much greater degree than they are currently, an education must first and 
foremost provide a student with knowledge and skills that will yield benefits for a lifetime, not 
simply a single job. 


Similarly, while job training cannot be the sole focus of a bachelor’s degree, there are far too 
many students who leave campus saddled with debt and lacking employable skills. In 
partnership with organizations representing students, taxpayers, educators, and colleges, we 
will support new paths to a meaningful credential and a fulfilling career. Through innovation and 
commonsense approaches to providing aid to those who need it most, we can make a quality 
postsecondary education more accessible and more affordable to all. 


When developing any new policy, the Department will remember the interests of students first. It 
will remember the service member leaving active duty and looking to build employable skills, the 
student with special needs looking to build a life of dignity and purpose but lacking the supports 
she deserves, and the child from poverty looking to build a better life but unsure where to turn 
for a helping hand. As the Department of Education begins a new chapter, it will be with a focus 
on empowering these and so many other Americans to realize their full potential. 


Data Verification and Validation 


The GPRA Modernization Act of 2010 requires agencies to prepare information on the reliability 
of data presented. OMB guidance indicates: 


Agencies may develop a single data verification and validation appendix used to 
communicate the agency’s approaches, and/or may also choose to provide information 
about data quality wherever the performance information is communicated (e.g., 
websites).? 


? OMB Circular A-11, Preparation, Submission, and Execution of the Budget, Part 6, Section 260.9, 2014. 
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The full data verification and validation summary and a high-level assessment of the 
completeness and reliability of the data presented are provided in appendix A of this report. 


Reporting on Progress 


The Department continues to use quarterly performance reviews, targeted strategic initiatives, 
and outreach to leaders and stakeholders to assess progress and garner engagement toward 
achieving strategic goals and outcomes. Continuous improvement rests on ongoing cycles of 
assessing performance, examining data, and applying lessons learned to improve practices. 
Creating a culture of continuous improvement is at the heart of the Department’s efforts to 
partner with and support educators, administrators, and policymakers, with the intent of 
obtaining better outcomes for all students. 


The Department's Strategic Plan portfolio of metrics (indicators) shifts year-over-year.° For 
example, metrics are retired as more meaningful metrics are identified that can drive the 
Department's strategic goals and objectives forward. In FY 2016, the Department added 
(including metrics re-baselined) a total of 18 performance metrics to its public annual 
performance reporting. The Department met 50 percent of the targets for those new metrics. 


Overall, in FY 2016, the Department met 27 of its 52 performance metric targets, exceeded the 
targeted results for 23 of those metrics, and just missed the targets for 7 metrics. The 
Department continues to be committed to identifying performance metrics that are meaningful 
and drive the Department’s work forward and to establishing targets that are both ambitious and 
attainable. With the removal and addition of performance metrics each fiscal year, the 
Department has seen a decrease in performance metric targets met overall from FY 2014 
through FY 2016. 


3 See appendix B (Changed Performance Goals) of the FY 2015 Annual Performance Report and FY 2017 Annual Performance 
Plan, available online at https://www2.ed.gov/about/reports/annual/2017plan/2015-2017-apr-app-plan-appdx-b.pdf. 
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External (Public-facing) Metrics 
= Metrics Met QTotal Metrics 


2016 52 
2015 46 
2014 48 


. Total External 
Fiscal . %  Year/Year 
External Metrics 
Year ; Me Change 
Metrics Met 


2014 31 


64.6% 
2015 46 31 67.4% 4.3% 


2016 52 27 51.9% -23.0% 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 13 


PERFORMANCE PLAN SUMMARY 


Goal 1. Postsecondary Education, Career and Technical Education, 
and Adult Education: 


Increase college access, affordability, quality, and completion by 
improving postsecondary education and lifelong learning 
opportunities for youths and adults. 


Goal Leader: Under Secretary 


Objective 1.1: Access and Affordability. Close the opportunity gap by improving the 
affordability of and access to college and/or workforce training, especially for underrepresented 
and/or underprepared populations (e.g., low-income and first-generation students, English 
learners, individuals with disabilities, adults without high school diplomas, etc.). 


Objective 1.2: Quality. Foster institutional value to ensure that postsecondary education 
credentials represent effective preparation for students to succeed in the workforce and 
participate in civic life. 


Objective 1.3: Completion. Increase degree and certificate completion and job placement in 
high-need and high-skill areas, particularly among underrepresented and/or underprepared 
populations. 


Objective 1.4: Science, Technology, Engineering, and Mathematics (STEM) Pathways. 
Increase STEM pathway opportunities that enable access to and completion of postsecondary 
programs. 


able) i (om =t-1ar=1i 15 


Increasing college access, affordability, quality, and completion by improving postsecondary 
education and lifelong learning opportunities for youths and adults requires attention to three 
equally important factors to facilitate success: availability of good consumer information and 
financial aid, strong motivation by students and families, and access to affordable, high-quality 
learning opportunities. 


In FY 2016, the Department delivered nearly $126 billion in grants, work-study, and loan 
assistance to more than 13 million postsecondary students at over 6,600 schools.‘ In addition, 
the Department administered $2 billion annually in grants to strengthen postsecondary 
institutions and promote college readiness, and nearly $1.7 billion in grant funds for career and 
technical education (CTE) programs® and adult education programs aimed at helping youth and 
adults attain the academic and technical skills and knowledge necessary to transition to 
postsecondary education, training, and employment. 


The Department developed resources, including the College Affordability and Transparency 
Center, the Financial Aid Shopping Sheet, and other loan counseling and financial literacy 
resources, to provide students and families with enhanced tools for informed decision-making. 
Additionally, the Federal Student Aid Feedback System provides a way for students, parents, 
borrowers, and others to file complaints about their experiences with federal aid programs and 


4 Federal Student Aid Annual Report FY 2016. 
5 Of the roughly $1.1 billion in Perkins formula funds that are distributed by states to local recipients, approximately 40 percent are 
allocated for postsecondary CTE programs. 
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institutions. The Department will use the feedback to improve the experience for current and 


future borrowers. 


Goal 1 Discretionary Resources 


FY 2018 


FY 2017 


FY 2016 $29,866 


$10,000 $20,000 


(Dollars in millions) 


$28,089 
$29,811 


$30,000 


WEN Colam DD tXorg-) tolat-l AVM aa cele]e-lutcw-lare Mm -VeltVit(-Soameele] o) ole) ai tate Me cor-| im 


Metrics [Dollars in Millions] 


$40,000 


edslaie)anitslares, 


FY 2017 FY 2018 
FY 2016 Annualized President’s 

POC Account Obj. Program Appropriation CR’ Budget 
FSA DM/SAA Student Aid Administration: Salaries and expenses 697 694 681 
FSA DM/SAA Student Aid Administration: Servicing Activities 855 855 1,017 
FSA SFA 1.1 Federal Pell grants: Discretionary 22,475 22,433 22,433 

1.1, ; ; 

OCTAE CTAE 12.13 Adult basic and literacy education state grants 582 581 486 
OCTAE  CTAE NA Career and technical education state grants 1,116 1,115 949 
OPE HE 1.1,1.3 Federal TRIO programs 900 898 808 
Subtotal 26,625 26,576 26,374 
Other Discretionary Programs/Activities 3,241 3,234 1,715 
TOTAL, GOAL 1 29,866 29,811 28,089 


POC = Principal Operating Component. 
CR = Continuing Resolution. 
NA = Not applicable. 


NOTES: Many programs may have sub-activities that relate to other goals. Detail may not add to total due to rounding. 


6 All the programs listed are discretionary programs, as distinct from mandatory programs. These include both competitive and 


noncompetitive/formula programs. 


7 A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 


the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 


provided by the continuing resolution. 
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© (oy-| fm eam BY=) 11 ky 


U.S. Department 


of Education Sau, Suva. Actual-to-Target 
Indicators of Actuals Year Year 2016 Out-Year Targets 
Success Baseline Target Results Trend Line 
Indicator y (Actuals) 
Measurement 2014 2016 2016 2016 Baa 2017 2018 
Direction Missed® Exceeded? 
FY: End of FY: 2013-14 FY: FY: 2015- | FY: 2015- MET 18.8% >= 3 
1.1.A. Federal FY 2014-15 24.0% 2014-15 16 16 % points 
student loan 21.7% 21.7% 19.4% 21.0% 2] Belew 30.0% 
delinquency rate previous 25.0% 
25.0% year’s actual | 20.0% 
aso 15.0% 
% ery 2016 are 
20.0% =a Target ae 
DECREASE ay oe 
gy aoe * Pace 2014 2015 2016 
10.0% 
09/12/2014— NA FY: 2015 FY: 2016 FY: 2016 NOT MET 1,800,000 TBD 
1.1.B. Web traffic 9/11/2015" 91,011 1,427,797 | 1,500,000 
to the College 91,011 1,525,000 peep 
Scorecard’® (Excluding Benalane 1,200,000 
new aaa, GFY 2016 1,050,000 
Scorecard 1,475,000 Target 909,000 
launch on 1,450,000 600,000 
INCREASE Sept. 12, ae MFY 2016 sega 
2015) cae Actual 150,000 
t 1,400,000 0 
2014 2015 2016 
1,375,000 


8 Missed target by <=1, or if percentage, <=1.3 percentage points. 
° Surpassed target; not just met the target. If a diminishing target, the actual was below the reduction target set. 
10 Performance metric is transitioning to Strategic Goal 5 in FY 2017. 
‘t In the 2015 APR, the baseline was noted as being a fiscal year. However, the baseline was measured using 09/12/2014—-09/11/2015 data. The period did not align with the fiscal 
year in order to exclude anomalous data from the release of the new College Scorecard on September 12, 2015. 
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U.S. Department 


; Current Current 
dima: Actuals Year Year gaa Out-Year Targets 
: Target Results Trend Line 
ures Baseline (Actuals) 
ndicator = 
Measurement 2016 2016 2016 ERS 2017 2018 
Direction Missed® Exceeded? 
1.1.C. Percentage SY: SY: SY: SY: 2015— | SY: 2015— | NOT MET NA NA 
of first-time 2012-13 2013-14 2014-15 16 16 65.0% 
FAFSA oe 59.2% 60.1% 60.5% 57.5% 59.5%— ee 65.0% 
among hig 61.5% 
school seniors’? ‘ 60.0% al Target te & “a 
57.5 mFY 2016 50.0% 
55.0% i 
INCREASE % petal 45.0% 
40.0% 
mate 2014 2015 2016 
New Metric: Application AY: 2014-15 AY: AY: 2016— NA NA NA 2,016,304 2,021,345 NA 
Nuniberar acct Cycle (FY: 2014) | 2015-16 17 
time FAFSA filers Applicants 2,021,691 (FY: (FY: 2016) 
among high 5-Year 2015) 2,015,138 
Average 2,009,155 
school seniors 012/13— 
2016/17 
2,016,304 


12 Revising metric language (in the 2014 and 2015 APR read as: “Percentage of high school seniors filing a FAFSA”) to align with how the metric is captured in the Federal Student Aid 
(FSA) Strategic Plan. All of the underlying assumptions and calculation methodologies were confirmed by FSA to be the same. Retiring metric at conclusion of FY 2016. Please see 
appendix B for additional information pertaining to the metric’s retirement. The FY 2017 target was 56.5%—58.5%, and FY 2018 was +/- one percentage point over prior year’s actual. 
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U.S. Department 
of Education 
Indicators of 

Success 


Baseline 


Indicator 
Measurement 
Direction 


1.1.D. Index of 
national 
aggregate annual 
earnings of VR 
consumers 
(based on the 
number of 
competitive 
integrated 
employment 
outcomes, hours 
worked, and 
hourly wages of 
VR consumers) 


FY: 
2010 
$57,971,317 


INCREASE 


t 


2014 


Actuals 


2015 


2016 


Current 
Year 
Target 


Current 
Year 
Results 


Actual-to-Target 
2016 


2016 


2016 


Missed? 


Exceeded? 


Out-Year Targets 


2017 


2018 


Trend Line 
(Actuals) 


FY: 2014 
$61,800,214 


FY: 2015 
$64,688,3 
9013 


FY: 2016 
$66,791,9 
33 


FY: 2016 
$65,608,8 
96 


MET 


67,000,000 


Wary 2016 
66,500,000 ey Target 


Sie mi FY 2016 


66,000,000 
Actual 


65,500,000 Aas 
, 
08,8 


65,000,000 96 


$66,921,074 


$68,259,495 


68,000,000 
66,000,000 
64,000,000 
62,000,000 
60,000,000 


58,000,000 
2014 2015 2016 


13 Metric was TBD in the 2015 APR due to data not being available until FY 2016. The metric was MET. 
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U.S. Department 
of Education 
Indicators of 

Success 


Baseline 


Indicator 
Measurement 
Direction 


1.1.E. Index of 
national 
aggregate annual 
earnings of 
Transition-Age 
Youth (based on 
the number of 
competitive 
integrated 
employment 
outcomes, hours 
worked, and 
hourly wages of 
VR Transition- 
Age Youth) 


FY: 
2010 
$15,971,665 


INCREASE 


t 


2014 


Actuals 


2015 


2016 


Current 
Year 
Target 


Current 
Year 
Results 


Actual-to-Target 
2016 


2016 


2016 


Missed? 


Exceeded? 


Out-Year Targets 


2017 


2018 


Trend Line 
(Actuals) 


FY: 2014 
$18,540,576 


FY: 2015 
$19,579,0 
744 


FY: 2016 
$19,980,9 
56 


FY: 2016 
$19,476,8 
18 


MET 


20,200,000 


@FY 2016 
Target 


20,000,000 

19,9 

Ce mi FY 2016 
Be Actual 


19,800,000 
19,600,000 
19,400,000 19,4 


76,8 
19,200,000 18 


$19,866,354 


$20,263,681 


20,500,000 
20,000,000 
19,500,000 
19,000,000 
18,500,000 
18,000,000 
17,500,000 
201420152016 


14 Metric was TBD in the 2015 APR due to data not being available until FY 2016. The metric was MET. 
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U.S. Department 


, Current Current 
dima: Actuals Year Year gaa Out-Year Targets 
: Target Results Trend Line 
ures Baseline (Actuals) 
ndicator ] 
Measurement 2016 2016 2016 eel 2017 2018 
Direction Missed? Exceeded? 
1.1.F. Number of | FY: 2009-14 NA FY: FY: 2015- | FY: 2015- MET 1516 TBD 
data points or Aggregate 2014-15 16 16 
other information 12 12 15 15 20 a 
reports released 15 15 
on the FSA Data oS GFY 2016 yo 
Center’® 15 ‘Target 10 
10 5 
mFY 2016 
Actual 0 
INCREASE 2014 2015 2016 
t 0 
AY: 2013-14 NA AY: AY 2014— AY 2014— NOT MET AY 2015-16 | AY: 2016-17 
1.2.A. Pell 24.1% 2013-14 15 15 WN 26.0% 27.0% 
enroliment at 24.1% 24.5% 25.0% = pon 
IHEs with high oie 
graduation rates” ane 24.0% —_ A 
% 23.0% 
@rY 2016 
INCREASE ae ee nes 
21.0% 
mFY 2016 20.0% 
Actual 2014 2015 2016 


t 


10.2% 


18 Metric is aligned with an Agency Priority Goal. 


16 Initial FY 2017 target of 30 provided in the 2015 APR was inclusive of FY 2016’s total of 15, thus making the total a cumulative 30. However, the actual FY 2017 target is 15 new 


releases. 


17 “High graduation rate” is defined as 65 percent or higher, which is roughly the 75th percentile. 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 


20 


PERFORMANCE PLAN SUMMARY 


U.S. Department 


ef Educaiion 7 Current Current Actual-to-Target 
Indicators of ctuals Year Year 2016 Out-Year Targets 
Success Baseline Target Results Trend Line 
Indicator = (Actuals) 
Measurement 2015 2016 2016 2016 ER 2017 2018 
Direction Missed® Exceeded? 
1.2.B. Number of 
states that FY: 2015 NA FY: 2015 FY: 2016 FY: 2016 MET 37 50 
develop or 8 8 20 10 BE 
strengthen career 
pathways 
policies, 2: GFY 2016 25 
guidance, or a Target - 
legislation FY 2016 ae 
a Actual 
INCREASE 10 a as 
5 
5 
0 
0 2014 2015 2016 
1.3.A. Degree Current CPS CPS CPS CPS NOT MET 48.4% 50.5% 
attainment Population Year: 2013 Year: Year: Year: = 
among 25-34- Survey 44.8% 2014 2015 2015 SS 
year-old age (CPS) Year: 45.7% 46.5% 46.8% 
cohort'® 2012 48.0% 46.8 47.0% 
44.0% me % 46.0% 
ate GFY 2016 45.0% 
INCREASE Target 44.0% 
44.0% 43.0% 
mFY 2016 42.0% 
Actual 41.0% 
42.0% 40.0% 
2014 2015 2016 
40.0% 


'8 Metric is aligned with an Agency Priority Goal. 
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U.S. Department 
of Education 
Indicators of 

Success 


Indicator 
Measurement 
Direction 


1.3.B. Enrollment 
in IHEs below the 
minimum 
earnings 
threshold'?”° 


DECREASE 


Baseline 


AY: 
2012-13 
9.7% 


Current Current 
Actuals Year Year gael Out-Year Targets 
Target Results Trend Line 
(Actuals) 
= 
2016 2016 ERE 2017 2018 
Missed® Exceeded? 
NA AY: AY: AY: NOT MET NA NA 
2012-13 2013-14 2013-14 11.5% 
9.7% 10.7% 9.4% eye 13.0% 
11.0% 
5% Target 
9.5% ie arge’ oe a 
mFY 2016 7.0% 
7.5% Actual 


5.5% 


5.0% 


3.0% 


2014 2015 2016 


'9 In the 2015 APR, the initial metric was identified as “Enrollment in IHEs where students’ median earnings 10 years after entering college are below a minimum earnings threshold.” 
The metric’s verbiage was condensed to “Enrollment in IHEs below the minimum earnings threshold.” Retiring metric at conclusion of FY 2016. Please see appendix B for additional 
information pertaining to the metric’s retirement. The FY 2017 target was 9.0 percent. 
20 “Minimum earnings threshold” is defined as the median earnings above the level of an institution at the 25th percentile for students 10 years after entering college, which equals 
$19,000 for less-than-two-year institutions, $26,000 for two-year institutions, and $35,000 for four-year institutions. 
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U.S. Department 
of Education Coa OTE Actual-to-Target 
F Actuals Year Year Out-Year Targets 
Indicators of 2016 : 
Success : Target Results Trend Line 
ind Baseline (Actuals) 
ndicator =] 
Measurement 2014 2015 2016 2016 2016 A eel ‘ 2017 2018 
Direction Missed Exceeded 
1.3.C. FY: 2015 FY: 2014 FY: 2015 FY: 2016 FY: 2016 MET 78.7%o— +/- one % 
Persistence 79.5%? 79.6% 79.5% 79.7% 78.5%- 80.7% point over 
among first-time 80.5% prior year’s 
filing aid Bras actual 
recipients”! ; 82.5% 
80.0% 80.0% —_——_ 
77.5% 
INCREASE ies si eg 75.0% 
78.0% ere: 725% 
70.0% 
77.0% mY 2016 67.5% 
Actual 65.0% 
76.0% 2014 2015 2016 
75.0% 
Leas Number of | ay. oo10-11 | AY:2011-12 | AY: | AY: 2013- | AY: 2013— | NOT MET | 625,000 637,672 TBD 
531,018 556,696 2012-13 14 14 
poste canealy 573,911 | 578,583 | 608,980 605,000 FY 2016 600,000 
credentials : ; ; Target 
awarded 585,000 580,000 
565,000 mFY 2016 560,000 , aie 
Actual 
INCREASE 545,000 540,000 
525,000 520,000 
2014 2015 2016 


21 Revising metric language (in 2015 APR read as: “FAFSA Renewal Rate”) to align with how the metric is captured in the Federal Student Aid (FSA) Strategic Plan. All of the 
underlying assumptions and calculation methodologies were confirmed by FSA to be the same. 
2 Modifying the baseline data to account for the actual being 79.5% (79.47%); however, the decimals were dropped by FSA when the metric was initially reported. 


https://studentaid.ed.gov/sa/sites/default/files/FY 2015 FSA Annual Report official.pdf 


23 FY 2016 target reduced from 691,000 to 637,672 due to a calculation error in how the targets were initially calculated. 
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Goal 1 FY 2016 Indicator Performance Summary 


14 
12 Total Indicators 


12 


10 
6 (50.0%) 


6 (50.0%) 


m@ Met m@ Not Met 


NA = Not applicable. 

TBD = To be determined. 

Academic Year (AY) is a collegiate year spanning August—May; School Year (SY) spans August—July and is aligned with a P—12 school year; Fiscal Year (FY) corresponds to a federal 
fiscal year; Calendar Year (CY) spans January—December. 


Da 
1.1 
1.1 
1.1 


ta Sources and Frequency of Collection: 


Federal Student Aid (FSA) Data Center; annually 

Google Analytics data from College Scorecard; annually 

The denominator is the number of graduating seniors according to the most recent projection by the National Center for Education Statistics (NCES) (Table 219.10 within the 
Digest of Education Statistics). The numerator is from FSA’s Central Processing System and is based on the number of applications during the first nine months of the 
application cycle that are—as of September 30 of the first year of the application cycle—complete (not rejected); first-time filers; incoming freshmen, with or without previous 
college attendance; age 18 or less as of June 30 of the first year of the application cycle; reporting high school diploma attainment; and attended a high school in the 

50 states and Washington, DC; annually 

Rehabilitation Services Administration-911 (RSA-911); annually 

RSA-911; annually 

FSA Data Center; annually 

Data from College Scorecard; annually 

Development data from the National Skills Coalition, possibly supplemental with data from the Association of State Legislators and Center on Law and Social Policy 
(CLASP); annually 

NCES Digest of Education Statistics, Table 104.30 (https://nces.ed.gov/programs/digest/d15/tables/dt15 104.30.asp), Number of persons age 18 and over, by highest level 
of educational attainment, sex, race/ethnicity, and age: 2015. Tabulated from Current Population Survey data, U.S. Census; annually 

Data from College Scorecard; annually 
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FSA’s Common Origination and Disbursement (COD) system; annually 


3.C 
A.A. IPEDS; annually 


Note on performance metrics and targets: These metrics were established as a part of the FY 2014-18 Strategic Plan. Metrics may be updated or revised to 
reflect awareness of more accurate data or clarifications. Such updates or revisions are identified in footnotes. 
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Analysis and Next Steps by Objective 


Objective 1.1: Access and Affordability. Close the opportunity gap by improving the 
affordability of and access to college and/or workforce training, especially for 
underrepresented and/or underprepared populations (e.g., low-income and first- 
generation students, English learners, individuals with disabilities, adults without high 
school diplomas, etc.). 


FY 2016 Implementation Strategy 


Although the Department has limited ability to directly impact college costs or control price, the 
Department made progress toward strategic objective 1.1. 


The Department’s implementation strategy in FY 2016 continued to focus on providing 
consumer tools and better information about postsecondary institutions to help students make 
more informed decisions; modifying and simplifying the Free Application for Federal Student Aid 
(FAFSA); expanding access to income-driven repayment plans; and improving customer service 
related to student aid. 


FY 2016 Barriers to Success 


Although many states are beginning to increase appropriations per full-time equivalent student, 
state-level funding for higher education overall remained low compared to levels before the 
recent recession. The Department had little influence over state funding decisions and was 
limited in its ability to impact affordability. Nonetheless, the Department will continue to explore 
ways to help ensure that borrowers are able to manage student loan debt. 


Key Milestones and Future Actions 


In 2016, the Department implemented two major changes to the FAFSA. The first change was 
to make the FAFSA available to students and families three months earlier than usual. In the 
past, the FAFSA was available January 1 for the upcoming financial aid award year (July 1 
through June 30). Beginning with the 2017-18 financial aid application cycle, the FAFSA 
became available October 1 of the year prior to the upcoming award year (2017-18 FAFSA 
filing began October 1, 2016). By having the FAFSA available earlier, the financial aid process 
generally will coincide with the college admissions application and decision cycle, allowing 
schools and states to provide financial aid information sooner to students and families. The 
second change involved the year for which tax information is collected on the FAFSA. Prior to 
FY 2016, when completing a FAFSA, applicants provided income information from the prior tax 
year (i.e., 2015 income information for the 2016—17 FAFSA). Beginning with the 2017-18 
FAFSA, income information from one tax year earlier—the so-called “prior-prior year’—will be 
collected. As a result of this change, students will be able to complete their FAFSA using 
information from an already-completed tax return. 


The Department continues to enhance the online FAFSA® Completion Tool, which helps 
financial aid professionals, school administrators, and guidance counselors track and increase 
FAFSA completion. The tool also provides FAFSA completion rates for school districts, allowing 
communities to tailor communications, support, and counseling to students while helping 
schools monitor their progress. 


Federal Student Aid (FSA) published reports about applications for the Public Service Loan 
Forgiveness (PSLF) Program, enrollment in income-driven repayment plans, and data on cohort 
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default rates. As of September 2016, nearly 5.8 million Direct Loan borrowers were enrolled in 
income-driven repayment plans—such as income-based repayment, Pay As You Earn (PAYE), 
and Revised Pay As You Earn (REPAYE)—a 38 percent increase from September 2015 anda 
110 percent increase from June 2014. Cohort default rates continued to decline from 14 percent 
in 2010 to 11 percent in 2013. 


Beginning in January 2016, the Department required all Direct Loan servicers to provide 
enhanced disclosures to borrowers and strengthened consumer protections. The recompetition 
of loan servicer contracts commenced in March 2016, with the goal of ensuring that servicers 
help borrowers responsibly make affordable monthly payments on their student loans. The final 
selection of a service is scheduled to be made and a contract should be awarded in FY 2017. 


Students, parents, and others can now provide feedback about their Title IV federal student aid 
experience via a streamlined, centralized portal, the Federal Student Aid Feedback System. 
Launched July 1, 2016, the system enhances customers’ ability to provide feedback and receive 
a timely and meaningful resolution. Customers can submit feedback about applying for and 
receiving federal loans, grants, and work study, as well as feedback about their experiences 
with federal loan servicers, collection agencies, and the Department itself. Complaints submitted 
through the feedback system are tracked from intake to resolution and will inform the continual 
improvement of the Department’s customers’ experience, regardless of where they are in the 
student aid process. Through the system, the Department has the ability to monitor the 
resolution process, intervene as needed, and compile data about complaints in order to improve 
the applicant and borrower experience. In FY 2017, the Department will publish the first annual 
report about feedback system data. 


The Department announced institutional participants in four experiments under the Experimental 
Sites Initiative conducted under FSA’s experimental sites authority. The Department announced 
participating institutions in the Dual Enrollment experiment in May, participating institutions for 
Second Chance Pell in June, participating institutions in Educational Quality through Innovative 
Partnerships in August, and participating institutions in the Loan Counseling experiment in 
December 2016. 


Objective 1.2: Quality. Foster institutional value to ensure that postsecondary education 
credentials represent effective preparation for students to succeed in the workforce and 
participate in civic life. 


FY 2016 Implementation Strategy 


The Department’s implementation strategy to address postsecondary quality included: using 
regulatory authority to strengthen program integrity; taking steps to strengthen accreditation 
oversight; fostering innovation by providing flexibility where appropriate, and testing ideas 
through experimental sites projects; encouraging the postsecondary community to focus on 
student learning that is validated through the assessment of actual learning outcomes; 
addressing teacher preparation initiatives to improve teacher quality; and promoting information 
about successful evidence-based programs and strategies. 


FY 2016 Barriers to Success 


The Department had limited authority and funding to promote innovative, evidence-based 
strategies and practices for college access and completion. 
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Key Milestones and Future Actions 


To improve accreditation transparency and promote outcome-driven accountability, the 
Department publishes a chart with each accrediting agency’s stated student achievement 
measures and key student and institutional metrics arranged by accreditor. The Department 
also requires accreditors to submit decision letters when they place institutions on probation, 
and the Department publicly posts releasable portions of such letters. The redesigned 
accreditation webpage better informs the public regarding the Secretary’s recognition process 
and provides a one-stop information repository for accreditation-related information. 
Accreditation staff at the Department now have regular access to critical outcomes data, state 
and federal litigation reports, and other information about each accreditor’s schools prior to 
conducting accrediting agency reviews. This information is also provided to the National 
Advisory Committee on Institutional Quality and Integrity. 


This year, the Department issued two letters to accreditors. The first, released in March 2016, is 
a “Terminology Letter,” which clarifies terminology and requirements for reporting to the 
Department. The second was an April 2016 “Flexibility Letter,” which clarifies and encourages 
accreditors to use flexibility to focus monitoring and resources on student achievement and 
problematic institutions or programs. 


The Department published several significant regulations. In October 2015, the Department 
published final regulations on debit cards to protect students from unscrupulous actions by 
financial institutions concerning student access to financial aid funds. Additionally, in August 
2016, the Department, in collaboration with the Department of Labor (DOL), published final 
regulations to implement the Workforce Innovation and Opportunity Act (WIOA). WIOA reforms 
the public workforce development system through enhanced coordination and collaboration 
across education and training programs, including employment and training programs 
administered by DOL, and adult education programs and vocational rehabilitation programs for 
individuals with disabilities administered by the Department. The final regulations assist states in 
implementation efforts, including fostering more cohesive planning within economic regions, 
enhancing performance accountability for education and employment outcomes, strengthening 
employer engagement, and improving access to education and workforce services for 
individuals with significant barriers to employment, among other efforts. The final regulations 
included program-specific rules related to adult education programs and vocational rehabilitation 
programs, as well as joint WIOA regulations related to unified and combined state plans, 
performance accountability, and the one-stop system. 


The Department published final regulations on Borrower Defense to Repayment in October 
2016 (effective July 1, 2017, with several provisions slated for early implementation) and State 
Authorization of Distance Education programs in December 2016, with implementation taking 
effect in July 2018. The Department published the Gainful Employment debt-to-earnings rates in 
January 2017 and will implement the rest of the requirements of the regulation by July 2017. 


In October 2015, the Department announced an initiative through FSA’s experimental sites 
authority to address alternative methods for quality assurance—dubbed Educational Quality 
through Innovative Partnerships, or EQUIP. The Department evaluated applications and in 
August 2016, invited eight institutions to continue to Phase 3 of the selection process. In 
Phase 3, these institutions will begin to set up their proposed programs and apply to the 
Department for final approval, which is expected during FY 2017. 


The Department conducted a number of grant competitions that incorporated evidence-based 
activities, including TRIO Talent Search, TRIO Educational Opportunity Centers, and several 
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programs for minority-serving institutions. In October 2015, the Department (the Office of 
Career, Technical, and Adult Education ((OCTAE]) awarded nine pilots under the Performance 
Partnership Pilots for Disconnected Youth (P3) program. The notice inviting applications (NIA) 
for round two (FY 2015) was published in April 2016, and for round three (FY 2016) in August 
2016. The agencies selected one pilot finalist for round two and six pilot finalists for round three. 
P3 tests the hypothesis that additional flexibility for states, localities, and tribes can help 
overcome some of the significant hurdles in providing intensive, comprehensive, and sustained 
service pathways and improving outcomes for disconnected youth. 


Objective 1.3: Completion. Increase degree and certificate completion and job placement 
in high-need and high-skill areas, particularly among underrepresented and economically 
disadvantaged populations. 


FY 2016 Implementation Strategy 


While the federal government has no direct impact on the educational attainment rate, the 
Department established annual targets since increasing the percentage of adults who have 
completed a college degree is a key Department objective. For the first time, the rate this year 
fell just short of the target by 0.3 percentage points. Future targets, set at progressively larger 
increases, may be difficult to achieve. 


With few tools to impact completion on a large scale, the Department’s implementation strategy 
focused on the ability to convene partners, disseminate noteworthy practices through reports, 
conduct research, and conduct prize challenges that support pathways in CTE. 


FY 2016 Barriers to Success 


The Department's ability to significantly impact completion rates nationwide depends on 
Congressional support for programs. Another factor affecting the Department’s ability to impact 
completion rates was the time lag between actions and changes initiated in the current year and 
results that will take several years to manifest as the current cohort moves through its multiyear 
educational program. 


Key Milestones and Future Actions 


In support of CTE, the Department announced winners of the CTE Makeover Challenge and the 
grand prize winner of the Reach Higher Career App Challenge. The Department also developed 
and published on November 2, 2016, an EdSim Challenge notice with the intent to announce 
winners in summer 2017. The challenge calls upon the gaming, developer, and educational 
technology communities to design simulated environments that prepare America’s students for 
a more competitive world through high-quality CTE. In September 2016, the Department 
awarded a $2 million Pay for Success (PFS) grant for providing technical assistance on the first 
two phases of a PFS project: a feasibility analysis in four local CTE sites, and transaction 
structuring for those local sites where a PFS project is deemed feasible. 


The Department participates in the Interagency Working Group on Supports for Postsecondary 
Success, which meets regularly to plan joint activities in support of access and completion for 
low-income students. 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 29 


PERFORMANCE PLAN SUMMARY 


Objective 1.4: Science, Technology, Engineering, and Mathematics (STEM) Pathways. 
Increase STEM pathway opportunities that enable access to and completion of 
postsecondary programs. 


FY 2016 Implementation Strategy 


The number of STEM postsecondary credentials awarded is shaped by actions taken by 
postsecondary institutions, by state and local agencies through funding decisions, and by 
market forces and job creation trends. Many external factors impact this objective, but the 
Department's STEM Team, within the Office of Innovation and Improvement (OIll), employs the 
following strategies to support this objective: 


e strengthening the Department’s existing STEM programs; 


e enhancing interagency collaboration focused on STEM by fostering stronger linkages 
across agencies that have STEM education programming and collaborating with other 
agencies; 


e using convenings, events, speeches, and other mechanisms to highlight models, 
interventions, and data, and share leading practices and policies for formal and informal 
STEM learning at the federal, state, and local levels; 


e partnering with nongovernment strategic partners to support STEM; and 


e identifying and helping to cultivate, disseminate, and encourage adoption of innovative, 
promising, and evidence-based practices to influence and shape the future of STEM 
education. 


FY 2016 Barriers to Success 


The STEM Team identifies programs within the Department and across the federal government 
that are well suited for enhancing and increasing STEM participation through strategic use of 
STEM priorities. For programs that have already implemented STEM strategies, the STEM 
Team lends its expertise for program review and evaluation. The investments at the Department 
that address STEM degree and credential completion in particular were limited to select 
programs that target minority-serving institutions. The Department will continue to promote 
STEM pathway opportunities within the Committee on STEM Education (CoSTEM) structure 
that includes community colleges, as well as engage with specific STEM-focused initiatives led 
by corporate and philanthropic entities that help elevate the quality of STEM programs and 
advance STEM participation, such as STEM Learning Ecosystems. 


Trends for females and minority students point to continued challenges in broadening 
participation in STEM. For example, in January, the National Science Foundation (NSF) 
National Center for Science and Engineering Statistics (NCSES) released the 2017 Women, 
Minorities, and Persons with Disabilities in Science and Engineering report, the federal 
government's most comprehensive look at the participation of these three demographic groups 
in science and engineering education and employment. The report found that while women 
have reached parity with men in educational attainment, they have not done so in employment 
in science and engineering. Furthermore, underrepresented minorities (black, Hispanic, and 
American Indian or Alaska Native) account for disproportionately smaller percentages in both 
science and engineering education and employment. 
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Key Milestones and Future Actions 


CoSTEM has established a task force which is working to enhance the undergraduate 
experience of STEM majors through a formally chartered interagency working group led by 
NSF. The group is focused on four major objectives: 


e evidence-based practices to improve undergraduate learning and retention in STEM; 


e community college efforts to both support two-year students and create bridges between 
two- and four-year postsecondary institutions; 


e research experiences that involve both university-industry and university-federal entity 
partnerships, particularly for students in the first two years; and 


e promoting mathematics success to help combat excessively high failure rates in 
introductory math courses at the undergraduate level. 


Representatives from the Department have been instrumental in bringing new focus to the role 
of community colleges and articulation programs in supporting undergraduate STEM education. 


Other areas of the Department are bolstering support for STEM, as well. STEM education is an 
integral component of CTE. OCTAE is leading initiatives seeking to increase knowledge of and 
access to postsecondary STEM opportunities. For example, the EdSim challenge is seeking 
next generation Virtual Reality and Augmented Reality to improve teaching in areas including 
STEM. As additional examples, the CTE makeover challenge will incorporate “making” and 
“maker spaces” into CTE programs by upgrading or modernizing facilities that meet the needs 
of manufacturing in the 21st century. The Reach Higher App challenge will spur innovation in 
career exploration by empowering students with individualized career and education 
information. And the White House Initiative on Historically Black Colleges and Universities 
(WHI-HBCU) convened events and panels in FY 2016 to share best practices across HBCUs for 
supporting and retaining students in STEM fields. 
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Goal 2. Elementary and Secondary Education: 


Improve the elementary and secondary education system’s ability to 
consistently deliver excellent instruction aligned with rigorous 
academic standards while providing effective support services to 
close achievement and opportunity gaps, and ensure all students 
graduate high school college- and career-ready. 


Goal Leader: Assistant Secretary, Office of Elementary and 
Secondary Education (OESE) 


Objective 2.1: Standards and Assessments. Support implementation of internationally 
benchmarked college- and career-ready standards, with aligned, valid, and reliable 
assessments. 


Objective 2.2: Effective Teachers and Strong Leaders. Improve the preparation, recruitment, 
retention, development, support, evaluation, recognition, and equitable distribution of effective 
teachers and leaders. 


Objective 2.3: School Climate and Community. Increase the success, safety, and health of 
students, particularly in high-need schools, and deepen family and community engagement. 


Objective 2.4: Turn Around Schools and Close Achievement Gaps. Accelerate 
achievement by supporting states and districts in turning around low-performing schools and 
closing achievement gaps, and developing models of next-generation high schools. 


Objective 2.5: STEM Teaching and Learning. Increase the number and quality of STEM 
teachers and increase opportunities for students to access rich STEM learning experiences. 


ad 0 le) (om ={:1ai=1i1! 


The goal for America’s elementary and secondary educational system is clear: all students 
should have meaningful opportunities to graduate from high school ready for college and a 
career. The important work in communities across the country over the past several years 

contributed to the highest ever national high school graduation rate, reaching 83.2 percent. 


However, while many schools are increasing the quality of instruction and improving academic 
achievement, our education system fails to consistently provide all students with equal access 
to a high-quality education, as evidenced by persistent achievement gaps between student 
subgroups. Data from the 2015 National Assessment of Educational Progress (NAEP) show 
that low-income students scored 24 to 28 points below their more advantaged peers in reading 
and math, respectively. The achievement gaps between black and white students were between 
24 and 32 points and achievement gaps between Hispanic and white students were between 

18 and 24 points. 


The Department’s elementary and secondary education programs focus on the building blocks 
needed for states, districts, and schools to more consistently deliver excellent classroom 
instruction for all students. The Elementary and Secondary Education Act (ESEA), as 
reauthorized by the ESSA, requires that all students in America be taught to high academic 
standards that will prepare them to succeed in college and careers and that vital information is 
provided to educators, families, students, and communities through annual statewide 
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assessments that measure students’ progress toward those high standards. The ESEA also 
promotes local innovation and the use of evidence-based interventions, particularly as part of 
locally determined efforts to turn around low-performing schools. 


Goal 2 Discretionary Resources 


FY 2018 


FY 2017 


FY 2016 


$0 $10,000 $20,000 $30,000 $40,000 
(Dollars in millions) 
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Metrics [Dollars in Millions] 


FY 2017 FY 2018 
FY 2016 Annualized President’s 

POC Account Obj. Program Appropriation CR Budget 
OESE ED 2.4 School improvement grants 450 449 -- 

OESE ED 2.1, 2.2, 2.3, 
2.4 Title | Grants to local education agencies 14,889 14,881 15,881 
Teacher and school leader incentive 
OESe ~!tl ma grants 230 230 200 
OESE SIP 25 Mathematics and science partnerships 153 152 _ 
OESE SIP 2.1 State assessments 378 377 377 
Supporting effective instruction state 

OESE SIP 2.2 grants 2,256 2,252 -- 
OESE SIP NA 21st century community learning centers 1,167 1,164 -- 
Oll 1&l 2.2, 2.3, 2.4 Charter schools grants 333 333 500 
Oll 1&1 2.2, 2.3, 2.6 Magnet schools assistance 97 96 96 
Oll SSCE 2.1,2.2,2.3 Promise Neighborhoods 73 73 60 
OSERS SE 2.1, 2.2, 2.3 Special Education grants to states 11,895 11,890 11,890 
Subtotal 31,959 31,898 29,005 
Other Discretionary Programs/Activities 2,448 2,444 1,871 
TOTAL, GOAL 2 34,407 34,342 30,876 


POC = Principal Operating Component. 

CR = Continuing Resolution. 

NA = Not applicable. 

NOTES: Many programs may have sub-activities that relate to other goals. Detail may not add to total due to rounding. 


24 All the programs listed are discretionary programs, as distinct from mandatory programs. These include both competitive and 
noncompetitive/formula programs. 

25 A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 
the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 
provided by the continuing resolution. 
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Goal 2: Details 


INCREASE 


t 


40 


U.S. Department of Current | Current 
Education A Year Year saa alge! Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement = 5 (Actuals) 
= 
Direction 2014 2015 2016 2016 2016 aE 2017 2018 
Missed? | Exceeded2” 
2.1.A. Number of SS 5 —_—y 
states/territories that have 52 52 50 
adopted college- and SY: SY: SNe 20 51 a 
career-ready standards® = 4545 | 2013-14 | 2014-15 | 2015-16 a3 @FY 2016 48 
"ig -| «51 (49 51 (49 51 sy: NOT 4g Target ie 
49, plus | PUSD-C. | plus D.C. | (49 plus | 2015-16 | jyer a4 Beycaie 52 50 “ 
INCREASE bc and and D.C. and 52 43 eet aan ane HIE 
Puerto Puerto Puerto 40 
Rico)?° Rico) Rico) 
2.1.B. Number of 
states/territories that are SY: 2012— 0 SY: SY: SY: NOT 82 50 
implementing next- 13 2014-15 2015-16 2015-16 MET 
generation reading and 0 49 (48 47 52 = 50 
mathematics plus DC) QFY 2016 40 
assessments, aligned with 50 Target 30 
college- and career-ready Brea 20 
30 10 
standards 45 7 Actual 0 


2014 2015 2016 


6 Missed target by <=1, or if percentage, <=1.3 percentage points. 
2” Surpassed target; not just met the target. If a diminishing target, the actual was below the reduction target set. 
8 The Department is no longer conducting ESEA Flexibility monitoring, but states continued to implement their ESEA Flexibility requests through August 1, 2016, before beginning the 
transition to the Every Student Succeeds Act (ESSA). 


2° Revising from “49, plus DC and Puerto Rico” reported in the 2015 APR to “51 (49 plus DC and Puerto Rico)” to be consistent with 2015’s language. 


30 The Department is no longer conducting ESEA Flexibility monitoring, but states continued to implement their ESEA Flexibility requests through August 1, 2016, before beginning the 
transition to the Every Student Succeeds Act (ESSA). 
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U.S. Department of Current | Current 
Education Actua Year Year seas alget Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement ee eae (Actuals) 
=] 
Direction 2014 2015 2016 2016 2016 ee 2017 2018 
Missed? | Exceeded?” 
2.2.A. Number of states 
that have fully SY: 2012- SY: SY: SY: SY: NOT NA NA 
implemented teacher and 13 2013-14 2014-15 2015-16 2015-16 MET 
principal evaluation and 6 7 8 8 22 : 
support systems that 25 7 — ios 
consider multiple 6 
measures of effectiveness, i eee 5 
with student growth as a 15 . : 
significant factor*' io mFY 2016 2 
Actual 1 
2 8 2014 2015 2016 
INCREASE a 
2.3.A. Disparity in the rates 
of out-of-school SY: 2011-— | SY 2013- Not TBD SY: TBD TBD NA 4.7 
suspensions for students 12 14 Collected SY 2014-15 Biennial % point 
with disabilities and youth 10.7 10.6 2014-15 6.7 Metric disparity | 110% —_—,, 
of color (youth of color % % point data % point —- 
metric) point disparity collected | disparity sae 
disparity in 2016 spi 
and “aL 
DECREASE available 6.0% 
in 2017 7 
2012 2014 


31 Retiring metric at conclusion of FY 2016. Please see appendix B for additional information pertaining to the metric’s retirement. The FY 2017 and 2018 targets were 39 and 42, 
respectively. 

82 The 2011-12 CRDC results could not be replicated. However, the Department is able to report the 2013-14 CRDC disparities for one or more out-of-school suspensions for K-12 
students (excluding 504-only students). The 2014 target was NOT MET. 
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U.S. Department of Current | Current 
Education A Year Year sii alget Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement = (Actuals) 
= 
Direction 2014 2015 2016 2016 2016 Lae 2017 2018 
Missed26 | Exceeded?’ 
2.3.B. Disparity in the rates 
of out-of-school SY: 2011- SY 2013- Not TBD SY: TBD TBD NA 1.2 
suspensions for students 12 14 Collected SY 2014-15 Biennial % point 
with disabilities and youth 5.7 6.6 2014-15 2.7 Metric | disparity | 7° es 
of color (SWDs, IDEA only % point % point data % point bee 
metric) disparity | disparity®? collected | disparity 
in 2016 3.0% 
and 
gis available 1.0% 
in 2017 2012 2014 
2.4.A. Number of 
persistently low SY: 2011- SY: SY: SY: SY: MET | BB 630 598 
graduation rate high 12 2012-13 | 2013-14 | 2014-15 | 2014-15 700 ee 
echoola™ 775 737 680 605 664 eine a | 
650 Target 
DECREASE aac = 
600 Actual a 
605 0 
ea 2014 2015 2016 
2.4.B. Percentage of SIG 
schools in Cohort 5 that SY: 2013- NA SY: SY: SY: MET 25.0% ee 23.3% 25.6% 
are above the 25th 14 2013-14 2014-15 | 2014-15 se % 0% 
percentile in mathematics, 19.7% 19.7% 21% 21.0% ory — 
as measured by their state 15.0% 2016 : ah 
assessments Toreee 20.0% 
10.0% 
Lay 15.0% 
Bre 2016 ; 
INCREASE Actual 
0.0% 10.0% 


2014 2015 2016 


33 The 2011-12 CRDC results could not be replicated. However, the Department is able to report the 2013-14 CRDC disparities for one or more out-of-school suspensions for K-12 


students (excluding 504-only students). The 2014 target was NOT MET. 
34 Metric is aligned with an Agency Priority Goal. 
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as their main assignment 
who hold a corresponding 
undergraduate degree*® 


INCREASE 


t 


U.S. Department of Current | Current vom 
Education A Year Year sai alget Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement = (Actuals) 
=] 
Direction 2014 2015 2016 2016 2016 Fe 2017 2018 
Missed? | Exceeded?’ 
Se oriecacaa || Siezos-i| ae sy: sy: sy: NOT 25.9% | 27.8% 
14 2013-14 2014-15 | 2014-15 MET oc* 
pibeerber i aaa 20.12% 20.1% | 19.5% | 23.0% FY nani 
percentile in ue noo eae cats 23.0% ae 20.0% sy 
reading/language arts, as Peel lee Target 19.0% 
measured by their state : al ee 18.0% 
assessments 19.0% a 2016 17.0% 
17.0% 5% Actual 16.0% 
INCREASE ea 
15.0% 2014 2015 2016 

2.5.A. Percentage of high =| sy: 2011- Not Not TBD 65.3% NA NA NA NA NA 
school and middle school 12 Collected | Collected | Q1 of FY 
teachers who teach STEM 62.2% 2018 


5 This is a quadrennial metric and based on data collection should not have had an FY 2017 target; thus the target of 65.3% identified in the 2015 APR has been removed. Retiring 
metric at conclusion of FY 2016. Please see appendix B for additional information pertaining to the metric’s retirement. 
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U.S. Department of 
Education 
Indicators of Success 


Indicator Measurement 
Direction 


2.5.B. Number of public 
high school graduates 
who have taken at least 
one STEM AP exam*® 


INCREASE 


t 


SY: 2011- 
12 
497,922 


Current | Current 
p Year Year spins ile Out-Year Targets 
Target Results Trend Line 
=] (Actuals) 
2014 2015 2016 2016 2016 Fe 2017 2018 
Missed26 | Exceeded?’ 
635,000 
SY: SY: SY: SY: NOT ory 691,541 | 759,381 | Grohe 
2013-14 2014-15 2015-16 | 2015-16 MET 630,000 2016 600,000 
555,119%” | 592,410° | 622,553 632,642 Target 585,000 
625,000 FY 570,000 
2016 555,000 
620,000 622, Actual psa 


553 


615,000 


525,000 
510,000 


2014 2015 2016 


36 Although the metric’s data has a lag in when it is reported, the cohort year, school year, and fiscal year align. The metric has been updated to reflect this alignment. 


37 In the 2014 APR, the performance target of 536,810 was reported as “Not Met.” However, it was “Met.” 
38 In the 2015 APR, the performance target of 581,419 was reported as “Not Met.” However, it was “Met.” 
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Goal 2 FY 2016 Indicator Performance Summary 


12 
10 Total Indicators 


10 


5 (50.0%) 


ya VAUR OYA) 


@Met MBNotMet TBD MNA 


NA = Not applicable. 
TBD = To be determined. 


Academic Year (AY) is a collegiate year spanning August—May; School Year (SY) spans August—July and is aligned with a P—12 school year; Fiscal Year (FY) corresponds to a federal 
fiscal year; Calendar Year (CY) spans January-December. 


Data Sources and Frequency of Collection: 


2.1.A. 
2.1.B. 
2.2.A. 
2.3.A. 
2.3.B. 
2.4.A. 
2.4.B. 


2.4.C. 


Elementary and Secondary Education Act (ESEA) Monitoring; annually 
ESEA Monitoring; annually 

ESEA Flexibility Applications and Monitoring; annually 

Civil Rights Data Collection (CRDC); biennially 

CRDC; biennially 

National Center for Education Statistics (NCES) EDFacts; annually 


Analytic dataset produced by the contractor for the SIG National Summary, because this provides an accurate list of SIG schools and flags for different exclusions that are 
included in the analysis. (The analytic dataset is a combination of EDFacts student achievement files in Math and Reading, the NCES Common Core of Data, SIG lists 
provided to EDFacts by OSS, and Exclusions that are generated by the contractor that apply to these results.); annually 


Analytic dataset produced by the contractor for the SIG National Summary, because this provides an accurate list of SIG schools and flags for different exclusions that are 
included in the analysis. (The analytic dataset is a combination of EDFacts student achievement files in Math and Reading, the NCES Common Core of Data, SIG lists 
provided to EDFacts by OSS, and Exclusions that are generated by the contractor that apply to these results.); annually 
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2.5.A. Schools and Staffing Survey (SASS), NCES; quadrennially 
2.5.B. College Board/Advanced Placement (AP) administrative records; annually 


Note on performance metrics and targets: These metrics were established as a part of the FY 2014-18 Strategic Plan. Metrics may be updated or revised to reflect awareness of 
more accurate data or clarifications. Such updates or revisions are identified in footnotes. 
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Analysis and Next Steps by Objective 


Objective 2.1: Standards and Assessments. Support implementation of internationally 
benchmarked college- and career-ready standards, with aligned, valid, and reliable 
assessments. 


FY 2016 Implementation Strategy 


Given that the ESSA made few changes to most of the assessment provisions under Title I, the 
Department moved ahead with its Title | Assessment Peer Review process in 2016, using the 
peer review guidance released in September 2015. Through this process, external peers are 
making recommendations to the Department regarding whether the state has sufficiently 
documented the quality of its assessment system and whether its assessments are consistent 
with the requirements under Title | and the peer review guidance and nationally accepted 
professional testing standards. Through the end of FY 2016, the Department reviewed 
components of 38 states’ assessment systems and began providing feedback to states in fall 
2016. 


The Department continued to make use of existing technical assistance resources, including the 
College and Career Readiness and Success Center, Center on Standards and Assessments 
Implementation, and Reform Support Network, to support state implementation. 


The ongoing work of the Office of Special Education and Rehabilitative Services (OSERS) 
through its Results Driven Accountability (RDA) is also a key activity supporting progress on this 
goal. RDA is shifting the Department’s accountability efforts from a primary emphasis on 
compliance to a framework that focuses on improved results for students with disabilities, while 
continuing to assist states in ensuring compliance with the Individuals with Disabilities Education 
Act’s (IDEA) requirements. RDA emphasizes child outcomes such as performance on 
assessments, graduation rates, and early childhood outcomes. 


FY 2016 Barriers to Success 


There are several external risks to achieving this strategic objective. During 2016, several state 
legislatures considered bills related to standards and assessments, including bills that would 
remove state standards or assessments that have been identified as college- and career-ready. 
While most of these bills did not move forward, states may reconsider this legislation in the 
future, particularly as states develop and implement plans as required under Title | of the ESSA. 


There also is a risk that implementation of college- and career-ready standards will not be 
successful at the local level and inadequate supports will be provided to teachers and students. 


Key Milestones and Future Actions 


The Department began the Title | assessment peer review process in 2016, and external peers 
reviewed documentation regarding approximately 38 state assessment systems. The 
Department began providing these states with feedback and a decision regarding the outcome 
of peer review in fall 2016 and will continue to provide feedback on a rolling basis throughout 
the winter of 2017. Historically, the majority of states are required to submit additional evidence 
after the initial peer review in order to demonstrate that their system meets all of the 
requirements of professional and technical testing standards, consistent with Title | of the ESEA. 
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Objective 2.2: Effective Teachers and Strong Leaders. /mprove the preparation, 
recruitment, retention, development, support, evaluation, recognition and the equitable 
distribution of effective teachers and leaders. 


FY 2016 Implementation Strategy 


The primary strategy the Department adopted for this objective is to support states and districts 
in the development and implementation of high-quality teacher and principal evaluation and 
support systems, as well as broader human capital management systems that use the results of 
evaluation systems to inform placement, retention, promotion, differential performance-based 
compensation, and other considerations. 


In FY 2016, the Department supported states in implementing educator evaluation and support 
systems and finalized approval of all states’ State Plans to Ensure Equitable Access to 
Excellent Educators (Educator Equity Plans). In addition to providing support through the 
Equitable Access Support Network (EASN), the Department hosted its inaugural Educator 
Equity Lab and worked to plan additional labs. The Department also issued guidance advising 
states that Educator Equity Plans remain in effect for both the 2015-16 and 2016-17 school 
years. 


FY 2016 Barriers to Success 


Prior to termination of the waivers of Title |, Part A granted through the ESEA Flexibility 
initiative, 42 states, the District of Columbia, and Puerto Rico committed to implementing 
educator evaluation and support systems and reporting their progress on implementing those 
systems to the Department. The ESSA, which was signed into law in the first quarter of 

FY 2016, terminated, effective August 1, 2016, the ESEA Flexibility waivers that had been 
granted to states. Given the change in law, states are no longer required to report to the 
Department the details of their evaluation systems and therefore the Department does not have 
a viable data source to collect data regarding states’ work to continue to implement these 
systems. 


Key Milestones and Future Actions 


During the fourth quarter of 2015 and the first quarter of 2016, the Department approved 
Educator Equity Plans for all 50 states, the District of Columbia, and Puerto Rico. The 
Department continues to work with states through the EASN to provide support as they work to 
implement their Educator Equity Plans. 


In March 2016, the Department cohosted the inaugural Educator Equity Lab with the Mississippi 
Department of Education. This full-day forum provided an opportunity for a wide variety of 
stakeholders to come together to carry forward the work embedded within Mississippi’s 
Educator Equity Plan. 


Nevertheless, the Department remained concerned about its progress against this metric 
throughout FY 2016, given the clear termination of the ESEA Flexibility waivers, as required by 
the ESSA on August 1, 2016. 


In FY 2017, the Department intends to host additional Educator Equity Labs and to continue to 
provide support for states through Office of State Support (OSS) program officers and the 
EASN, as well as to support states in implementing the requirements under ESEA, as 
reauthorized by the ESSA, Section 1111(g)(1)(B) and 34 CFR 299.1 8(c). 
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The Department anticipates reviewing and revising its strategies for meeting the Effective 
Teachers and Strong Leaders objective to reflect the changes made by Congress in the ESSA. 


Objective 2.3: School Climate and Community. /ncrease the success, safety, and health 
of students, particularly in high-need schools, and deepen family and community 
engagement. 


FY 2016 Implementation Strategy 


The Department, broadly, and the Office of Elementary and Secondary Education (OESE), 
specifically, play a role in improving academic achievement, equity in education, and other 
important youth outcomes by working to understand and improve conditions for learning in 
public schools. Students’ academic achievement and their eventual success in school and in life 
are sensitive to the broader context in which they live and learn, and in which their schools 
operate. 


OESE’s implementation strategy included supporting activities related to the improvement of 
student success and school safety, discipline, health, and climate with a focus on districts and 
schools receiving School Improvement Grants (SIG). 


Examples of activities that supported progress towards this strategic objective include: 


e OESE’s Office of Safe and Healthy Students (OSHS), in coordination with the Office of 
the Deputy Secretary, sponsored two regional “Rethink Discipline” convenings. 


e On July 27, 2016, the Department, in partnership with the Department of Health and 
Human Services (HHS) and the Centers for Medicare & Medicaid Services (CMS), 
sponsored the Healthy Students, Promising Futures (HSPF) Learning Collaborative, 
bringing together teams from 10 states to work on expanding school health services for 
Medicaid-enrolled and eligible students by leveraging CMS’ change to the free care 
policy. The Learning Collaborative builds on the HSPF guidance and toolkit jointly 
released by the Department and HHS in January 2016. 


FY 2016 Barriers to Success 


Limited resources are a risk to achieving this strategic objective, such as identifying funds to 
support new efforts to provide additional technical assistance to improve state and local 
systems of support for chronically absent students. 


Key Milestones and Future Actions 


Future actions to support conditions for student learning in order to improve student attendance 
and achievement in the nation’s schools include: 


e Supporting states and local communities in identifying and meeting the needs and 
aspirations of chronically absent students. To extend this work, the Department is 
currently supporting the National Student Attendance, Engagement, and Success 
Center, which provides states and local communities access to expert assistance in 
establishing early warning prevention and intervention systems that aim to link 
chronically absent students with supportive services to improve student attendance and 
youth success in school and in life. 


e Supporting states and local communities in improving access to school-based health 
services, particularly for low-income and vulnerable youth. 
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e Supporting states and local communities in improving school safety and climate through 
school discipline reform and investments in assessing, measuring, and responding to 
school climate issues at state, local, and school levels. This includes funding grant 
programs directed to SEAs and local educational agencies (LEAs), as well as a technical 
assistance center that provides training, disseminates resources, and responds to 
inquiries. 


Objective 2.4: Turn Around Schools and Close Achievement Gaps. Accelerate 
achievement by supporting states and districts in turning around low-performing 
schools and closing achievement gaps, and developing models of next-generation high 
schools. 


FY 2016 Implementation Strategy 


The Department's efforts to turn around schools were largely focused on the distribution of SIG 
funding and technical assistance to states. The Department awards grants to states, which then 
award competitive subgrants to school districts. 


In schools that have received funds under the SIG program, up to 80 percent of students are 
from low-income families—28 percentage points higher than the average school.°? 


The State Support Network is a technical assistance center that supports state and district 
efforts to achieve significant improvements in student outcomes, scale up effective systemic 
approaches and practices within and across states and districts, and identify and share effective 
practices to facilitate learning from states, districts, and others to support school improvement. 
The Department also continued to partner with the Corporation for National and Community 
Service to support the School Turnaround AmeriCorps program grantees, and partnered with 
the President’s Council on Arts and Humanities to support the Turnaround Arts Initiative. 


FY 2016 Barriers to Success 


Turning around the lowest-performing schools is challenging work and takes several years to 
show progress and success. In addition, as states implemented new college- and career-ready 
standards and assessments aligned with those standards, it was challenging to measure 
progress over time using the mathematics and reading/language arts assessments. 


In addition, under the ESSA, states will still have financial resources to devote to school 
improvement efforts. As such, the Department will ensure a continuity of support to the field as 
states transition to the implementation of the law. 


Key Milestones and Future Actions 


All FY 2015 and FY 2016 SIG formula funds were awarded to states that applied for funds. The 
Department also utilized multiple existing technical assistance efforts to support states in 
implementing SIG. These efforts will continue in the future to support states as they transition to 
implementing the ESSA. 


3° http://www2.ed.gov/programs/sif/signationalsum09292015.pdf 
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Objective 2.5: STEM Teaching and Learning. Increase the number and quality of science, 
technology, engineering, and mathematics (STEM) teachers and increase opportunities 
for students to access rich STEM learning experiences. 


FY 2016 Implementation Strategy 


In lieu of budget appropriations for proposed STEM initiatives, the Department worked to 
strengthen existing programs that have a focus on STEM, enhance interagency collaborations, 
propose new areas of focus and work with external organizations to build public-private 
partnerships to increase the number and quality of STEM teachers and increase opportunities 
for students to access rich STEM learning experiences. 


STEM was included as a competitive or invitational priority in many discretionary grant 
competitions in FY 2016, including the Magnet School Assistance Program (MSAP), Investing in 
Innovation Program (i3), Hispanic Serving Institution STEM Articulation Program (HSI-STEM), 
and Ready to Learn (RTL). In addition, states, districts, schools, and their partners may utilize 
formula dollars to support STEM education. Examples of ways that SEAs, LEAs, and their 
partners could use formula funds (under Title I, Il, Ill, and IV of the ESEA; IDEA; and the Car! D. 
Perkins Career and Technical Education Act of 2006) to support STEM were provided in Q2 of 


2016: https://www2.ed.gov/programs/promiseneighborhoods/stemdearcolleaqgueacces.pdf. 


In addition to supporting existing Department programs and proposing new areas of focus in 
STEM, the Department has worked closely with the National Science and Technology Council 
CoSTEM that coordinates federal programs and activities in support of STEM education 
pursuant to the requirements of Sec. 101 of the America COMPETES Reauthorization Act of 
2010.*° The Department cochairs the P-12 and Computer Science for All Interagency Working 
Groups and actively participates in the Engagement Interagency Working Group. 


The Department also expanded interagency partnerships through the 21st Century Community 
Learning Centers (21st CCLC) program, a $1.1 billion formula grant program. The funds are 
used to provide high-quality, hands-on out-of-school learning experiences that connect learning 
that takes place during the school day with real-world applications in STEM areas. Since 2013, 
the partnerships have grown in scale and scope from two agencies—the Department and the 
National Aeronautics and Space Administration (NASA)—reaching 20 sites across three states 
to five agencies—the Department, NASA, the National Park Service, the Institute of Museum 
and Library Services, and the National Oceanic and Atmospheric Administration—reaching 
more than 200 sites across 25 states in 2016. 


In addition, the Department has worked closely with a number of partners, such as 100Kin10, a 
network of over 280 partners that came together in response to the President's 2011 call to 
action, in the State of the Union address, to recruit 100,000 STEM educators in the next 

10 years; in May 2016, 100Kin10 announced that they have the commitments in hand to recruit 
100,000 additional STEM teachers by 2021. Another successful partnership is with the STEM 
Funders’ Network around their STEM Learning Ecosystems initiative, which has cultivated 

37 community-based partnerships nationwide focused on providing high-quality STEM 
education opportunities, both in and out of school, for students from underserved and high-need 
communities. These partners have undertaken remarkable work to increase the number and 
quality of STEM educators and expand opportunities for students in STEM. 


4° https://www.nsf.gov/statistics/about/BILLS-111hr5116enr.pdf 
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FY 2016 Barriers to Success 


The ESSA eliminated Mathematics and Science Partnerships (MSP), the Department’s program 
to improve elementary and secondary school mathematics and science teacher education and 
professional development. With the elimination of MSP, the ESSA does not authorize any 
Department program dedicated solely to improving K-12 STEM education. If funded, the Title IV 
block grant may be used partially for STEM activities; but it is spread across multiple aspects of 
well-rounded education, and since the funds are distributed by formula, funding for LEAs for 
STEM education would likely be minimal. Limited resources present a risk to achieving this 
strategic objective, as STEM programming requires funds to support recruitment, training, 
support, and retention of STEM educators. 


Key Milestones and Future Actions 


In March 2016, STEM leaders across the country, representing state and local entities, 
foundations, nonprofits, media organizations, technology companies, research institutions, and 
museums, made commitments to support innovative STEM work. Collectively, these 
commitments have the potential to bring new, active STEM content for the nation’s youngest 
children to millions of households across the nation. In addition to the public and private sector 
groups that stepped up, federal agencies are deepening the resources and support they provide 
for early active STEM learning. 


The Department also will review and revise its strategy for supporting STEM Teaching and 
Learning to reflect changes made by the Congress in the ESSA. 
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Goal 3. Early Learning: 


Improve the health, social-emotional, and cognitive outcomes for all 
children from birth through 3rd grade, so that all children, 
particularly those with high needs, are on track for graduating from 
high school college- and career-ready. 


Goal Leader: Assistant Secretary, Office of Elementary and 
Secondary Education (OESE) 


Objective 3.1: Access to High-Quality Programs and Services. Increase access to high- 
quality early learning programs and comprehensive services, especially for children with high 
needs. 


Objective 3.2: Effective Workforce. Improve the quality and effectiveness of the early learning 
workforce so that early childhood educators have the knowledge, skills, and abilities necessary 
to improve young children’s health, social-emotional, and cognitive outcomes. 


Objective 3.3: Measuring Progress, Outcomes, and Readiness. Improve the capacity of 
states and early learning programs to develop and implement comprehensive early learning 
assessment systems. 


Public Benefit 


Each year, about 4 million children enter kindergarten in the United States. Unfortunately, not 
every parent has access to high-quality early learning opportunities for their children. There are 
large disparities in enrollment based on geography, race, and socioeconomic levels. As a result, 
too many children enter kindergarten a year or more behind their classmates in academic skills 
and social-emotional development." 


A robust body of research shows that children who participate in high-quality preschool 
programs have better health, social-emotional, and cognitive outcomes than those who do not 
participate. The gains are particularly powerful for children from low-income families and those 
at risk for academic failure who, on average, start kindergarten 12 to 14 months behind their 
peers in preliteracy and language skills.** Studies also reveal that participating in quality early 
learning can boost children’s educational attainment and earnings later in life.*° 


In FY 2016, the Department’s investments in early learning included IDEA Part C and Preschool 
Development Grants (PDG). 


41 Yoshikawa, Hirokazu, Christine Weiland, Jeanne Brooks-Gunn, Margaret R. Burchinal, Linda M. Espinosa, William T. Gormley, 
Jens Ludwig, Katherine A. Magnuson, Deborah Phillips, and Martha J. Zaslow. Investing in Our Future: The Evidence Base on 
Preschool Education. Vol. 9. Society for Research in Child Development and Foundation for Child Development, 2013. 

42 Committee on Integrating the Science of Early Childhood Development. From Neurons to Neighborhoods: The Science of Early 
Childhood Development. (2000). Jack P. Shonkoff and Deborah A. Phillips, eds. Board on Children, Youth, and Families, 
Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. 

48 See, e.g., Yoshikawa, et.al., Investing in Our Future: The Evidence Base on Preschool Education, 2013. 
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Goal 3 Discretionary Resources 


FY 2018 


FY 2017 


FY 2016 


$1,000 


$1,500 


(Dollars in millions) 


$2,000 


$2,500 


WN FeNroyam BJ CXevg-) tColar-lavm manele le-liitcm- lave M-Vel th aid {-t-mmameole] o) Lelailale Meror-] MM m-vaelgiir-lare-) 
Metrics [Dollars in Millions] 


FY 2017 FY 2018 
FY 2016 Annualized President’s 

POC Account Obj. Program Appropriation CR*® Budget 
OESE 1&l 3.1 Preschool development grants 250 250 - 
OSERS SE 3.1, 3.2, 3.3 Grants for infants and families 459 458 458 
OSERS SE 3.1, 3.2, 3.3 Preschool grants 368 368 368 
Subtotal 1,077 1,075 825 
Other Discretionary Programs/Activities 26 26 0 
TOTAL, GOAL 3 1,103 1,100 825 


POC = Principal Operating Component. 
CR = Continuing Resolution. 


NOTES: Many programs may have sub-activities that relate to other goals. Detail may not add to total due to rounding. 


4 All the programs listed are discretionary programs, as distinct from mandatory programs. These include both competitive and 


noncompetitive/formula programs. 


45 A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 
the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 


provided by the continuing resolution. 
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(© (oy: | eam BY=1 = 11 () 


U.S. Department of 
Education 
Indicators of Success 


Indicator Measurement 
Direction 


3.1.A. Percentage of 4-year 
olds enrolled in state 
preschool programs® 


INCREASE 


t 


SY: 
2015-16 
(FY: 2016) 
85.5% 


Current Current 
Vear Year saa arget = 
Target Results Trend Line 
= (Actuals) 
2014 2015 2016 2016 2016 [Be 2017 2018 
Missed*® | Exceeded‘” 
NA NA SY: NA NA NA 92.7% 100% NA 
2015-— 
16 
(FY: 

2016) 
85.5% 


46 Missed target by <=1, or if percentage, <=1.3 percentage points. 
47 Surpassed target; not just met the target. If a diminishing target, the actual was below the reduction target set. 
48 Metric is aligned with an Agency Priority Goal (APG). This metric is being revised from what was reported in the 2015 Annual Performance Report (APR) to focus on the 

18 Preschool Development Grants (PDG)-funded states (Alabama, Arizona, Arkansas, Connecticut, Hawaii, Illinois, Louisiana, Maine, Maryland, Massachusetts, Montana, Nevada, 
New Jersey, New York, Rhode Island, Tennessee, Vermont, Virginia) and more accurately reflect the grants’ impact. The data source has also been changed from National Institute 
for Early Education Research (NIEER) Yearbook to the states’ annual performance reports: Preschool Development Grants Annual Performance Reports, Data Workbook, Table A(1) 


(a) and (b). 
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U.S. Department of Current Current Actualto-raraet 
Education Actua Year Year 2016 9 O ar Targ 
Indicators of Success : Target Results Trend Line 
P = (Actuals) 
cee ee 2014 2015 | 2016 | 2016 2016 a 2017 | 2018 
Direction Missed*® | Exceeded*’ 
3.1.B. Number of states : : : : 10 
Hae : SY: SY: sy: SY: TBD TBD 12 TBD 
with high-quality SY: 2013- ie 8 
preschool program 14 rire 4 | 2014-15 | 2015- | 2015-16 ; aA 
standards”? (FY: 2014) | 944) (FY: 16 (FY: 
: a | 3” | aor | 40 
INCREASE Q3 of 2 
Follow- . 
t ing FY 2014 2015 2016 
3.2.A. Number of states SY: 2013- SY: SY: SE SY: TBD TBD 18 TBD a 
that require a teacher with 14 2013-14 2014-15 2015- 2015-16 —as, 
a bachelor’s degree ina (FY: 2014) (FY: (FY: 16 (FY: sd 
state preschool program™ 115 2014) 2015) (FY: 2016 - 
11 11° 2016) 15 6 
TBD 4 
Q3 of 2 
INCREASE Follow- 0 
ing FY 2014 2015 2016 


49 Metric is aligned with an APG. This metric is being revised from what was reported in the 2015 APR to focus on the 18 PDG-funded states (Alabama, Arizona, Arkansas, 
Connecticut, Hawaii, Illinois, Louisiana, Maine, Maryland, Massachusetts, Montana, Nevada, New Jersey, New York, Rhode Island, Tennessee, Vermont, Virginia) and more 
accurately gauge the grants’ impact. The data source is being updated to reflect the column from which the data are pulled: NIEER Yearbook Table 1: State Ranking and Quality 
Checklist Sums, Column “Quality Checklist Sums.” 


°° This metric is being revised to focus on the 18 PDG-funded states (Alabama, Arizona, Arkansas, Connecticut, Hawaii, Illinois, Louisiana, Maine, Maryland, Massachusetts, 
Montana Nevada, New Jersey, New York, Rhode Island, Tennessee, Vermont, Virginia) and more accurately gauge the grants’ impact. All of the data have been revised from what 


was reported in the 2015 Annual Performance Report. The data source is being updated to reflect the column from which the data are pulled: National Institute for Early Education 
Research (NIEER) Yearbook, Workforce Supplement, Table 3: Preschool Teacher Qualifications and Salary, Column “BA Required.” 

51 For the baseline year/data, the NIEER SY 2013-14 Yearbook did not have a Workforce section, which is where the data for this metric should be pulled. Thus the “Teacher has a 
BA’ column was used and the 11 states that met the criteria were: Alabama, Connecticut, Illinois, Louisiana, Maine, Maryland, Nevada, New Jersey, New York, Rhode Island, and 
Tennessee. 

52 In SY 2014-15 (FY 2015) the column changed to “BA Required,” so if a state reported on more than one program, and all programs did not require a BA, then the state was not 
included in the count for the state requiring a teacher with a bachelor’s degree in a state preschool program. ere a state reported on more than one program, all programs in that 
state must require a BA in the state preschool program to be included in the data count. PDG-funded states meeting criteria include: Alabama, Hawaii, Illinois, Louisiana, Maine, 


Maryland, Nevada, New Jersey, New York, Rhode Island, and Tennessee. 
FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 51 


PERFORMANCE PLAN SUMMARY 


U.S. Department of Current Current von 
Education Actua Year Year seis arget O ear Targe 
Indicators of Success Ph Ns Target Results Trend Line 
i sy (Actuals) 
sia! Measurement 2014 2015 2016 2016 2016 Ba 2017 | 2018 
irection Missed“® | Exceeded4” 
3.3.A. Number of states SY: 2010 SY: SY: SNe SY: MET 22] 1655 TBD*® 56 
collecting and reporting (FY: 2010) | 2013-14 2014-15 2015- 2015-16 30 
disaggregated data on the 2 (FY: (FY: 16 (FY: 17 15 
status of children at 2014) 2015) (Eye 2016) 15 14 as 
kindergarten entry using a 5°8 1154 2016) 14 gry 2016 
common measure 17 10 Target : 
0 
5 mFY 2016 
INCREASE eae 2014 2015 2016 
a 0 


53 Five ELC states implemented their KEA (OR, KY, VT, MD, and OH) in the 2014-15 school year. One state (DE) had planned to implement its KEA in 2014—15 year, but later 
adjusted its timeline to implement during the 2015-16 school year. As such, the FY 2014 actual is revised from six to five states. 

54 Eleven ELC states (CA, CO, DE, KY, MD, MA, MI, NC, OH, OR, and VT) are implementing their KEAs in the 2015-16 school year. The remaining eight states that chose to 
implement KEAs (GA, IL, MN, NJ, NM, PA, RI, WA) will begin after the 2015-16 school year. Wisconsin did not select to implement a KEA, but is implementing a statewide literacy 
assessment and is exploring the development and use of a KEA. 

55 There will be difficulty collecting ELC data in out-years because some grantees will no longer be reporting APR data. 

56 In FY 2017, the data source will change from RTT-ELC to EDFacts. However, the data will not be available until FY 2018 Q3 with a collection commencement timeframe of 
December 2017. Nine (9) of 19 RTT-ELC states that wrote to this optional area will have completed their grants and the Department will only have 10 states reporting in FY 2017. KEA 


data collection has been added to EDFacts. 
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Goal 3 FY 2016 Indicator Performance Summary 


5 
4 Total Indicators 


4 
3 
2 

1 (25.0%) 
1 

1 (25.0%) 
0 


SNA mMet TBD 


NA = Not applicable. 

TBD = To be determined. 

Academic Year (AY) is a collegiate year spanning August—May; School Year (SY) spans August—July and is aligned with a P—12 school year; Fiscal Year (FY) corresponds to a federal 
fiscal year; Calendar Year (CY) spans January-December. 


Data Sources and Frequency of Collection: 

3.1.A. Preschool Development Grants Annual Performance Reports, Data Workbook, Table A(1)(a) and (b); annually 

3.1.B. National Institute for Early Education Research (NIEER) Yearbook Table 1: State Ranking and Quality Checklist Sums, Column “Quality Checklist Sums”; annually 
3.2.A. | NIEER Yearbook, Workforce Supplement, Table 3: Preschool Teacher Qualifications and Salary, Column “BA Required”; annually 

3.3.A. Race to the Top (RTT)-Early Learning Challenge (ELC) Technical Assistance Center; annually 


Note on performance metrics and targets: These metrics were established as a part of the FY 2014-18 Strategic Plan. Metrics may be updated or revised to 


reflect awareness of more accurate data or clarifications. Such updates or revisions are identified in footnotes. 
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Analysis and Next Steps by Objective 


Objective 3.1: Access to High-Quality Programs and Services. Increase access to high- 
quality early learning programs and comprehensive services, especially for children with 
high needs. 


FY 2016 Implementation Strategy 


In 2010, the Department and HHS formed an Early Learning Interagency Policy Board (IPB) to 
advise the Secretaries on how to better align programs and systems, and improve coordination 
and administration of federally funded early learning programs serving children from birth to 
age 8 (e.g., Head Start, Child Care, Home Visiting, and early learning programs funded under 
Title |, Part A of the ESEA and IDEA). Meeting quarterly, the IPB has provided a venue for 
developing policy recommendations and improving program coordination and quality across 
federally funded early learning and development programs. Through the work of the IPB, 
several policy statements were released that focus on improving the quality of early childhood 
programs. 


Improving access to high-quality early learning programs has been a priority for the Department. 
In FY 2016, the key programs on which the Department focused are the PDG and IDEA 
programs serving children from birth through age 5. Other efforts to improve the quality of early 
childhood programs focused on activities such as reducing suspensions and expulsions, 
addressing chronic absenteeism, promoting family engagement and inclusive settings, and 
supporting STEM for early learners. 


Preschool Development Grant Technical Assistance (PDG TA)—the technical assistance 
provider for the PDG grantees—held a number of webinars in FY 2016, providing information on 
communicating with families to prevent chronic absenteeism, supporting young children 
demonstrating difficult behaviors, and avoiding suspension and expulsion in early childhood 
programs. 


In order to improve the quality of early learning programs from birth through third grade, the 
Department and HHS collaborated with the National Center for Education Research (NCER) at 
the Institute of Education Sciences (IES) and leveraged $20.5 million of NCER resources with 
$5.5 million of PDG national activities funds to support the $26 million Early Learning Research 
Network. 


Several webinars and documents were produced in FY 2016 to provide the tools and resources 
to support families and their engagement with children’s learning and development including 
resources on supporting children and families living in poverty in rural areas and family 
engagement. 


FY 2016 Barriers to Success 


As PDG program administration transitions to HHS under the ESSA, the Department and HHS 
will work together to continue to support existing grantees and ensure that they are able to 
successfully meet their commitments during the project period. 


Despite a recent increase in federal funding, IDEA Part C systems within many states were 
stretched thin because of high demand for early intervention services. Some states have 
narrowed their definition of developmental delay as part of their eligibility requirements for 
serving children with disabilities and their families. Staff and Department-funded centers support 
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states in using their resources most efficiently. This was the first time in many years that there 
was an increase in Part B, Section 619 funding, yet because there has been an increase in the 
number of children served in the program over the years, the funding increase did not 
substantially increase the historic per child funding for the program. 


Key Milestones and Future Actions 


OSERS and its technical assistance centers have been supporting states on implementing and 
evaluating their State Systematic Improvement Plans (SSIPs) to improve results for children 
with disabilities. As OESE, OSERS, and HHS developed policy statements, the Department 
conducted input sessions with multiple stakeholder groups to ensure their perspectives were 
included and to get their input on additional resources that would be of benefit. 


In October 2016, the Department and HHS held their annual meeting for the 29 Race to the 

Top — Early Learning Challenge (RTT-ELC) and PDG grantees—11 RTT-ELC and 18 PDG—to 
share best practices and lessons learned. In February 2017, the Institute of Medicine/National 
Research Council (NRC) released a new report, “Fostering School Success for English 
Learners: Toward New Directions in Policy, Practice, and Research,” and in July 2016, 
“Supporting the Parents of Young Children” was released. Both were funded by the Department, 
HHS, and philanthropic partners. 


The Department also may consider revising this objective and/or strategies for achieving the 
objective to reflect changes to early learning programs made by the Congress in the ESSA. 


Objective 3.2: Effective Workforce. Improve the quality and effectiveness of the early 
learning workforce so that early childhood educators have the knowledge, skills, and 
abilities necessary to improve young children’s health, social-emotional, and cognitive 
outcomes. 


FY 2016 Implementation Strategy 


A number of activities were held to support states in building and strengthening the early 
learning workforce, including a joint webinar with HHS and a Peer Learning Exchange entitled, 
“Cultivating Excellence: Developing and Strengthening Your Early Childhood Workforce.” 


The new PDG program under the ESSA will provide continued support for the early learning 
workforce. The Department participated in meetings and planning regarding implementation of 
the new PDG under the ESSA, including working with HHS, OMB, and the Domestic Policy 
Council on creating an agreement between the Department and HHS to implement the program. 
The nonrequlatory guidance developed by the Department will help ensure that administrators, 
teachers, and paraeducators are supported as they work to improve outcomes for young 
children in early learning programs. 


In addition, OSERS funds the Early Childhood Personnel Center (the Center) to work with 
states to improve their personnel systems to increase the quality and effectiveness of the early 
learning workforce to serve children with disabilities. The Center recently finished providing 
intensive technical assistance to a cohort of four states, targeted technical assistance to six 
states, and chose six more states for intensive technical assistance. The Center also held 
leadership institutes for IDEA Part C and Part B, Section 619 coordinators. OSERS also 
continues to fund personnel preparation grants. The Center was supplemented to develop 
online modules on evidence-based practices for use by faculty and professional development 
providers preparing personnel that will serve young children with disabilities and their families. 
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On October 27, 2015, the Department and HHS released a report that summarizes the progress 
of professional development systems in the RTT-ELC states. In November 2015, the Early 
Learning Challenge Technical Assistance Program (ELC TA) released a report, Articulation 
Strategies in RTT-ELC States, to highlight best practices in supporting future practitioners in 
transferring from two-year colleges to four-year universities. ln December 2015, ELC TA 
released a report, RTT-ELC Grantees That Incorporate Business Trainings for Child Care 
Providers in Their Scope of Work. In January 2016, ELC TA published a report, Scholarships for 
Early Childhood Educators in RTT-ELC States. In June 2016, the Department and HHS 


released Credentialing in the Early Care and Education Field Report. 


FY 2016 Barriers to Success 


While the ESSA includes language explicitly stating that Title II dollars can be used for early 
childhood educators, it remains unclear whether or how that will affect the workforce. 
Challenges abound in developing an effective early learning workforce. States that can sustain 
programs for longer periods have less difficulty recruiting and retaining strong early childhood 
educators. 


In FY 2016, the Department and its technical assistance providers worked to address some of 
these challenges through webinars, peer learning, and highlighting promising practices, such as 
mentoring and coaching. 


Limited resources are a risk to achieving this strategic objective. The IDEA Part C and Part B, 
Section 619 programs report personnel shortages, specifically related to service providers who 
have specialized knowledge and skills in serving young children with disabilities and their 
families (e.g., speech-language pathologists, physical therapists). Additionally, there is an 
increased demand for resources under Part D that can address personnel shortages and 
improve the quality of the workforce serving young children with disabilities and their families. 


Key Milestones and Future Actions 


During May 2016, the Office of Early Learning (OEL) worked with HHS to develop a plan for the 
National Academies of Science to review and synthesize available research and analysis on the 
resources needed to meet the true costs of high-quality early care and education. The 
committee will produce a report that synthesizes the information gathered and, based on their 
analysis and interpretation, draw conclusions about and make recommendations for concrete, 
implementable funding strategies in the public and private sectors at the national, state, and 
local levels. In addition, an early childhood workforce case study is progressing well. 


During September 2016, the Department’s Policy and Program Studies Service (PPSS) office 
worked with HHS to develop a plan for a study on articulation from two-year to four-year 
colleges. The American Institutes for Research (AIR) is contracted to complete the study and is 
supported by PDG national activities funds. The report is expected to be released late next year. 


The Department may also consider revising this objective and/or strategies for achieving the 
objective to reflect changes to early learning programs made by the Congress in the ESSA as 
well as related budget proposals by the new administration. 
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Objective 3.3: Measuring Progress, Outcomes, and Readiness. /mprove the capacity of 
states and early learning programs to develop and implement comprehensive early 
learning assessment systems. 


FY 2016 Implementation Strategy 


The Department will continue to reach out to the Education Commission of the States (ECS), 
the Council of Chief State School Officers (CCSSO), the National Institute for Early Education 
Research (NIEER), and other organizations that share its interest in advancing quality 
kindergarten entry assessments (KEAs), share resources, and develop strategies that might 
increase its collective impact. 


OESE staff are working with Enhanced Assessment Grants (EAG) staff in supporting the three 
EAG grantees developing KEAs, paying particular attention to the RTT-ELC and PDG 
overlapping states. The Department has reviewed the applications submitted for the PDG 
competition that discuss the states’ assessment practices, and will use the PDG TA contractor’s 
electronic grant monitoring tool, GRADS 360, and other means to monitor and report on 
KEA/assessment progress. OESE is working with the national comprehensive center, CEELO, 
to provide targeted technical assistance on KEA development or enhancement. 


The Department, HHS, and ELC TA support both the RTT-ELC states and many voluntary 
nongrantee states and PDG states through an online early learning community. In October 
2015, the ELC TA center updated its brief, “Kindergarten Entry Assessments in ELC Grantee 
States.” 


On August 27, 2015, the Department posted for comment the ED Facts Data Set “Kindergarten 
Entry Assessment Data Collection through EMAPS’” as part of the Annual Mandatory Collection 
of Elementary and Secondary Education Data. The Department is reviewing and responding to 
comments received and expects to post the final data collection in FY 2017. The data collection 
will fill a void left when RTT-ELC KEA data collection diminishes as states finish their grant 
periods. 


To support implementation of the high-quality preschool program components, the Department 
and HHS reviewed the PDG and RTT-ELC APR data collected through February 2016. The 
PDG TA center published a brief, “State Technical Assistance Report: Comprehensive 
Assessment Systems in Preschool Development and Expansion Grant States.” In September 
2016, the ELC TA center updated its report, Kindergarten Entry Assessments in RTT-ELC 
States. Project officers continue to work with grantees to support implementation of high-quality, 
coordinated early learning systems and continued monthly calls with grantees to provide 
technical assistance. 


FY 2016 Barriers to Success 


Some states have challenges implementing their KEAs due to funding constraints and local 
district decisions. For example, in Maryland, one of three EAG grantees, a new law restricts the 
Maryland Kindergarten Readiness Assessment to sampling, rather than assessing all children 
when they enter kindergarten. A district may still voluntarily choose to assess all its 
kindergarteners. 


Key Milestones and Future Actions 


On October 27, 2015, the Department and HHS released a report that shows the Department 
surpassed the 2015 performance target of nine states collecting and reporting disaggregated 
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data on the status of children at kindergarten entry using a common measure. Eleven ELC 
states (CA, CO, DE, KY, MD, MA, MI, NC, OH, OR, and VT) are implementing their KEAs in the 
2015-16 school year. The remaining eight states that chose to implement KEAs (GA, IL, MN, 
NJ, NM, PA, RI, and WA) will begin after the 2015-16 school year. 


The 2015 APRs for 19 states—W1 did not choose to spend funds on a KEA—show that all 
states have made progress in moving towards implementing their KEAs. The 18 PDG states are 
required to report on the school readiness of the children participating in their high-quality 
preschool programs, with encouragement to use a KEA. States describe the assessment they 
will use in the 2016 APR to report the school readiness of the first PDG cohort of children. The 
Department expects to have data in spring 2017 for the second year of the grant. The 18 states 
reported serving 28,000 4-year-olds in high-quality preschool programs. In the 2016 APR for 
PDG, the Department will collect data on school readiness. 


The individual state RTT-ELC APRs and Progress Reports were posted on September 1, 2016. 
Seventeen of the 19 states that used RTT-ELC funds for KEAs are either phasing in 
implementation (7 states) or fully implementing (10 states) their KEA in the 2016-17 school 
year. Rhode Island and Georgia are piloting their KEA. 


The Department also may consider revising this objective and/or strategies for achieving the 
objective to reflect changes to early learning programs made by the Congress in the ESSA as 
well as the policy priorities of the new administration. 
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Goal 4. Equity: 


Increase educational opportunities for underserved students and 
reduce discrimination so that all students are well-positioned to 
succeed. 


Goal Leader: Assistant Secretary for Office for Civil Rights (OCR) 


Objective 4.1: Equitable Educational Opportunities. Increase all students’ access to 
educational opportunities with a focus on closing achievement gaps, and remove barriers that 
students face based on their race, ethnicity, or national origin; sex; sexual orientation; gender 
identity or expression; disability; English language ability; religion; socioeconomic status; or 
geographical location. 


Objective 4.2: Civil Rights Compliance. Ensure educational institutions’ awareness of and 
compliance with federal civil rights obligations and enhance the public’s knowledge of their civil 
rights. 


Public Benefit 


Equal access to equitable opportunities in education permeates every facet of the Department’s 
work and is the cornerstone of the mission to promote student achievement and preparation for 
global competitiveness. Through grant programs, loans, technical assistance, and civil rights 
enforcement, the Department improves educational opportunities and outcomes for all 
students—regardless of income, home language, ZIP code, age, sex, sexual orientation, gender 
identity, race, or disability. 


The Department works to serve students from their very first instructional experiences through 
early learning investments and to serve P-12 students through the implementation of the ESSA, 
which was signed into law in December 2015 and goes into effect with the 2017-18 school year. 
The ESSA continues the ESEA’s longstanding focus on providing resources and supports for 
students from low-income families, students with disabilities, English learners, and students who 
are migrant, homeless, or in foster care. The ESSA also focuses on providing support and 
interventions for the lowest-achieving schools as well as providing equal access to excellent 
educators for low-income and minority students. 


In addition to enhancing educational opportunities, the Department also works to eliminate 
discriminatory barriers that might prevent students from achieving their fullest potential. One 
way that the Department continues to monitor progress toward closing equity gaps in the 
nation’s schools is through the biennial CRDC. In FY 2016, the Department released its 
2013-14 CRDC report, demonstrating that despite significant work from districts across the 
country, persistent disparities remain, highlighting the need for a continued focus on educational 
equity. Another way is through continued efforts in OCR to address issues of equity in 
educational opportunity through both its policy and robust enforcement work. 
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Goal 4 Discretionary Resources 


FY 2018 


FY 2017 


FY 2016 


$1,000 $1,500 
(Dollars in millions) 


$2,000 $2,500 


WN FNroyam BJ CXevg-) tColar-lavm manele le-liitcw- lave M-Vel th Zid {-t-pmmeod i] o) oLelailale MCLor-] Wm n-vaceleiir-larer-) 
Metrics [Dollars in Millions] 


FY 2017 FY 2018 
FY 2016 Annualized President’s 
POC Account Obj. Program Appropriation CR® Budget 
OCR OCR Office for Civil Rights 107 107 107 
OESE ED 4.1 State agency programs: Migrant 375 374 374 
Indian Education: Grants to local 
Oree 7 NA educational agencies 100 100 100 
Indian Education: Special programs 
aici a oe puiiancaienc 38 38 38 
OESE SIP NA Alaska Native education 32 32 -- 
OESE SIP NA Native Hawaiian education 33 33 = 
OESE SIP 4.1,4.2 Training and advisory services 7 7 7 
OESE/OELA ELA 4.1,4.2 English Language Acquisition 737 736 736 
OSERS SE NA Special Olympics education programs 10 10 -- 
TOTAL, GOAL 4 1,440 1,437 1,362 


POC = Principal Operating Component. 


CR = Continuing Resolution. 
NA = Not applicable. 
NOTES: Many programs may have sub-activities that relate to other goals. Detail may not add to total due to rounding. 


57 All the programs listed are discretionary programs, as distinct from mandatory programs. These include both competitive and 
noncompetitive/formula programs. 
58 A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 
the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 


provided by the continuing resolution. 
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(© yoy-| fr- m BY=1F-11 ks 


U.S. Department of Current Current 
Education A Year Year aia Ai Bigot Out-Year Targets 
Indicators of Success : Target Results Trend Line 
i = (Actuals) 
a 2014 2015 2016 2016 2016 WS ail 2017 2018 
4.1.A. National high school SY: SY: SY: SY: FY: NOT MET | Ry 85.0% 85.3% ; 
graduation ratet! 2011-12 | 2012-13 | 2013-14 | 2014-15 2016 — — 
80.0% 81.4% 82.3% | 83.2% | 84.5% 80.0% 
85.0% == 84.5 75.0% 
% 
INCREASE 80.0% +MEEWMe ERY 2016 70.0% 
75.0% Pad Target 65.0% 
FY 2016 pete 
zoo S Actaal 2014 2015 2016 
65.0% 
4.1.B. Gap in the 
raduation rate between SY: SY: SY: SY: FY: MET a 7.4% 6.8% 
students from low-income | 2913-14 | 2012-13 | 2013-14 | 2014-15 ave 9.5% 2.0% a 
families and all students 7.7% 8.1% 7.7% 7.1% 7.6% 2 5.0% 
5.0% 
7 | tl Gry 2016 ae 
DECREASE eam == Tareet 10% 
5.5% % eI 
mFY 2016 = 
Actual 2014 2015 2016 
3.5% 


5° Missed target by <=1, or if percentage, <=1.3 percentage points. 


6° Surpassed target; not just met the target. If a diminishing target, the actual was below the reduction target set. 


5! Metric is aligned with an Agency Priority Goal (APG). 
6 SY 2014-15 actuals are being used to compare against the FY 2016 target; SY 2015-16 (aligns with FY 2016) data not available until 2017. 


®3 Metric is aligned with an APG. 


4 SY 2014-15 actuals are being used to compare against the FY 2016 target; SY 2015-16 (aligns with FY 2016) data not available until 2017. 
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U.S. Department of Current Current 
Education A Year Year spk A Out-Year Targets 
Indicators of Success : Target Results Trend Line 
5 = (Actuals) 
ee 2014 | 2015 | 2016 | 2016 2016 sca 2017 | 2018 
Direction Missed®? | Exceeded® 
4.1.C. Number of schools 
that do not have a gap or SY: 2013- NA SY: SY: FY: NOT MET | 13,350 13,442 13,487 
that decreased the gap 14 2013-14 | 2014-15 | 2016 nee si 
between students from 13,048 13,048 13,158 13,244 13,150 a 
low-income® families and Eo) 
the state average of all 13,050 | ar oe sale 
students® ° ae 12,750 
12,950 
mFY 2016 12,550 
INCREASE Hey Actual 2015 2016 
t 12,750 
4.2.A. Percentage of 25.0% 
proactive civil rights FY: 2013 FY: 2014 FY: Ee FY: MET FRE 15.0% 15.0% ; 
investigations launched 7.0% 21.0% 2015 2016 2016 ae sal AN 
annually that address 16.0% 20.0% 12.0% Gry 2016 15.0% 
areas of concentration in 20.0% Target 10.0% 
civil rights enforcement® 15.0% | omy FY 2016 sie 
; Actual ee 
GE oe eS: 220 2014 2015 2016 
5.0% ” 


t 


55 For purposes of this metric, eligibility for Free or Reduced Price Lunches (FRPL) under the National School Lunch Program (NSLP) is the primary source of data for identifying 
economically disadvantaged (low-income) students for reporting on student outcomes, including graduation rates. The Department is currently considering options for redefining 
“economically disadvantaged” students for student outcomes reporting and other uses. Should the Department make such a change, data on economically disadvantaged students 


may not be entirely comparable with data for previous years. 


86 Metric is aligned with an Agency Priority Goal (APG). 
87 This measure is calculated as the number of schools in which the ACGR for low-income students was equal to or greater than the statewide ACGR for all students PLUS the number 
of schools with a gap that reduced the size of this gap by 5 percent or more. Initially the data points in the 2015 APR were for a percentage calculation versus the number. 
68 SY 2014-15 actuals are being used to compare against the FY 2016 target; SY 2015-16 (aligns with FY 2016) data not available until 2017. 
8° Retiring metric at conclusion of FY 2016. Please see appendix B for additional information pertaining to the metric’s retirement. If a new metric is being proposed, the new metric will 
be directly below the indicator measurement direction of the metric being retired. 
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U.S. Department of Current Current 
Education ANG ITE: Year Year spe A Out-Year Targets 
Indicators of Success ee Target Results Trend Line 
= (Actuals) 

ee 2014 | 2015 | 2016 | 2016 2016 sca 2017 | 2018 

Direction Missed*? | Exceeded® 
New Metric: Average FY: 2016 NA NA FY: NA NA NA 5.27 5.32 NA 
number of cases 5.17 2016 
substantively” resolved, 5.17 
per investigative staff 
member 
4.2.B. Percentage of FY: 2013 FY: FY: FY: FY: MET Ba 16.0% 18.0% 
proactive civil rights 8.0% 2014 2015 2016 2016 aoe ene 
investigations resolved 15.0% 20.0% 33.0% 12.0% gry 2016 30.0% 
annually that address 30.0% Target 
areas of concentration in i leis 
civil rights enforcement”! 20.0% mFY 2016 10.0% 

Actual 
10.0% 00% 
INCREASE oe 2014 2015 2016 
t : 

New Metric: Number of FY: 2016 NA NA FY: NA NA NA 333 341 NA 
technical assistance 317 2016 
presentations and press 317 
releases on Office for Civil 
Rights’ work 


70 “Substantively” resolved includes cases with the following outcomes: finding of insufficient evidence, closure through early complaint resolution (ECR) process, cases that resulted in 
change without an agreement, and cases that resulted in change with a resolution agreement. The ultimate disposition of a case (that is, the case outcome) is tracked in OCR’s 
database and includes the above categories. For example, an outcome that is considered not substantive would be a dismissal. 

™ Retiring metric at conclusion of FY 2016. Please see appendix B for additional information pertaining to the metric’s retirement. If a new metric is being proposed, the new metric will 
be directly below the indicator measurement direction of the metric being retired. 
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Goal 4 FY 2016 Indicator Performance Summary 


6 
5 Total Indicators 


4 2 (40.0%) 


3 (60.0%) 


mw Met m@ Not Met 


NA = Not applicable. 

TBD = To be determined. 

Academic Year (AY) is a collegiate year spanning August—May; School Year (SY) spans August—July and is aligned with a P—12 school year; Fiscal Year (FY) corresponds to a federal 
fiscal year; Calendar Year (CY) spans January-December. 


Data Sources and Frequency of Collection: 

4.1.A. National Center for Education Statistics (NCES) EDFacts; annually 

4.1.B. NCES EDFacts; annually 

4.1.C. NCES EDFacts; annually 

4.2.A. Office for Civil Rights’ (OCR) Case Management System (CMS) and Document Management (DM) systems; quarterly 
4.2.B. OCR CMS and DM systems; quarterly 


Note on performance metrics and targets: These metrics were established as a part of the FY 2014-18 Strategic Plan. Metrics may be updated or revised to 
reflect awareness of more accurate data or clarifications. Such updates or revisions are identified in footnotes. 
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Analysis and Next Steps by Objective 


Objective 4.1: Equitable Educational Opportunities. Increase all students’ access to 
educational opportunities with a focus on closing achievement gaps, and remove 
barriers that students face based on their race, ethnicity, or national origin; sex; sexual 
orientation; gender identity or expression; disability; English language ability; religion; 
socioeconomic status; or geographical location. 


FY 2016 Implementation Strategy 


The Department is committed to pursuing equity at all stages of education, from birth through 
adulthood, by supporting institutions of: early learning; elementary and secondary education; 
career and technical, and postsecondary education; adult education; workforce development; 
and independent living programs. The Department's goal is to ensure that all—not just a 
subset—of the nation’s children have access to high-quality preschool, graduate high school, 
and obtain the skills necessary to succeed in college, in the pursuit of a meaningful career, and 
in their lives. Accordingly, the equity goal incorporates programs and initiatives across the 
Department. 


In December 2015, Congress passed the ESSA, which reauthorized and amended the ESEA. 
The ESSA continued the ESEA’s longstanding commitment to equal opportunity for all students 
with its focus on ensuring that students from low-income families and students of color have 
equitable access to excellent educators and its requirement that meaningful actions are taken to 
improve the lowest-performing schools. Throughout FY 2016, the Department worked diligently 
to analyze the changes to the ESEA made by the ESSA and provide guidance and technical 
assistance to states, districts, and the public on the new law, as well as on the transition to the 
new law. The Department coordinated this support across offices, including OESE, OSERS, the 
Office of English Language Acquisition (OELA), Oll, and the Office of Planning, Evaluation and 
Policy Development (OPEPD). 


In addition to supporting states as they prepare to implement the ESSA, the Department also 
worked to improve equitable access by removing discretionary barriers to education. In 

FY 2016, OCR continued its work to enhance equitable opportunity for students through the 
development of civil rights guidance materials and by enforcing federal civil rights laws to 
remove discriminatory barriers to education. 


OCR and the Office of the General Counsel (OGC) continued to support the equity goal of 
increasing educational opportunities for underserved students and reducing discrimination by 
representing the Department in litigation. In FY 2016, OGC worked with attorneys from OCR 
and the Department of Justice (DOJ) to file 13 statements of interest and amicus curiae briefs in 
federal courts to clarify the government's interpretation of civil rights laws. 


FY 2016 Barriers to Success 


A key challenge is the continued implementation of the changes to the ESEA made by the 
ESSA in addition to managing the transition from the No Child Left Behind Act (NCLB). Limited 
resources are a risk to achieving this strategic objective, and pose challenges to 
implementation. 


As SIG ends, limited capacity at the state, district, and school levels could impact the 
sustainability of reforms in schools and support for the implementation of school-based 
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interventions. Ensuring quality and completeness of data at the state and local levels to enable 
better measurement of success also remains a challenge. 


Key Milestones and Future Actions 


Staff will continue to support states on their Educator Equity Plans. In FY 2017, the Department 
intends to host additional Educator Equity Labs and to continue to provide support for states 
through OESE program officers and the EASN. 


On October 11, 2016, the Department announced awards to expand opportunity in CTE and 
dual language programs under the PFS Initiative. Additionally, the Department awarded a 
technical assistance grant to the Boston-based Social Finance Inc., in partnership with Jobs for 
the Future, in order to improve outcomes for underserved, high-need youth, through the 
development of PFS projects to implement new or scale up existing high-quality CTE 
opportunities. The Department also awarded a contract to the Washington, DC-based AIR to 
identify effective strategies to improve outcomes for children learning English. The study 
focuses on early learning-dual language programs for English learners from pre-K to grade 3. 


The Department also may consider revising its strategies for achieving the objective to reflect 
changes to programs made by the Congress in the ESSA as well as the policy priorities of the 
new administration. 


Objective 4.2: Civil Rights Compliance. Ensure educational institutions’ awareness of 
and compliance with federal civil rights obligations and enhance the public’s knowledge 
of their civil rights. 


FY 2016 Implementation Strategy 


OCR’s implementation strategy for this strategic objective involves the issuance of policy 
guidance, robust data collection, vigorous enforcement through investigations and monitoring, 
proactive technical assistance and engagement with stakeholders through interagency working 
groups, and the dissemination of information and response to public inquiries to enhance the 
public’s knowledge of their civil rights. In the policy arena, in FY 2016, OCR issued five policy 
guidance documents. 


In FY 2016, OCR unveiled its 2013-14 CRDC, published a First Look document detailing 
preliminary findings, and, for the first time ever, released the full, privacy-protected data set for 
the 2013-14 CRDC, which is available for direct download from the Department’s webpage. 
OCR continues to work with other offices in the Department to prepare and publish data sheets 
based on the CRDC 2013-14 data findings, including the rollout of chronic absenteeism data 
and website, which were published in June. 


In FY 2015, OCR overhauled its website to increase usability and to provide more information to 
the public, and in FY 2016, OCR continued to update its website with case resolution 
agreements and letters, policy guidance documents, technical assistance materials, and 
information about OCR’s enforcement processes. 


OCR continued to provide excellent customer service to enhance the public’s knowledge of their 
civil rights by responding to public inquiries for information. Through the Customer Service 
Team, OCR responded to 5,025 incoming correspondence inquiries and answered 8,019 OCR 
“Hotline” call inquiries in FY 2016. The Department also distributed 701 copies of selected OCR 
publications in response to inquiries from advocacy groups, educational institutions, state and 
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local educational agencies, parents, students, members of the general public, and enforcement 
offices. 


FY 2016 Barriers to Success 


In FY 2016, OCR received a record-high number of complaints (16,720) and resolved 

8,631 complaints despite challenges facing its operations, including a massive long-term 
staffing shortage and compressing and moving office space. Complaint volume and limited 
resources impacted OCR’s ability to conduct and resolve proactive investigations and to 
conduct proactive technical assistance. OCR continues to leverage all resources, both staffing 
and technological, to ensure that its enforcement and outreach efforts are timely, efficient, and 
effective, and used the minimal budgetary relief provided in FY 2016 to recruit, hire, and 
on-board new staff to support the more than 60 percent increase in complaint volume. 


Key Milestones and Future Actions 


In FY 2017, OCR will continue its focus on improving the quality and efficiency of investigations 
through training, technology, innovation, and strategic partnerships; expanding transparency; 
increasing proactive efforts to highlight the full range of OCR’s work through the release of data 
and materials to the public; and expanding technical assistance available to the public. In 
December 2016, OCR released its FY 2016 Annual Report, detailing efforts to protect students’ 
civil rights. 
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Goal 5. Continuous Improvement of the U.S. Education System: 


Enhance the education system’s ability to continuously improve 
through better and more widespread use of data, research and 
evaluation, evidence, transparency, innovation, and technology. 


Goal Leader: Assistant Secretary, Office of Planning, Evaluation and 
Policy Development (OPEPD) 


Objective 5.1: Data Systems and Transparency. Facilitate the development of interoperable 
longitudinal data systems for early learning through employment to enable data-driven, 
transparent decision-making by increasing access to timely, reliable, and high-value data. 


Objective 5.2: Privacy. Provide all education stakeholders, from early childhood to adult 
learning, with technical assistance and guidance to help them protect student privacy while 
effectively managing and using student information. 


Objective 5.3: Research, Evaluation, and Use of Evidence. Invest in research and evaluation 
that builds evidence for education improvement; communicate findings effectively; and drive the 
use of evidence in decision-making by internal and external stakeholders. 


Objective 5.4: Technology and Innovation. Accelerate the development and broad adoption 
of new, effective programs, processes, and strategies, including education technology. 


ed 0 le) (om =<:1ai=Vi1! 


Education stakeholders, ranging from students and parents, to teachers and principals, to 
institutional leaders and the Secretary, need access to timely, appropriate, relevant, and 
actionable information. Sources of helpful education information can range from datasets to 
rigorous evaluations and research studies to consumer-oriented tools. They must be accessible 
in multiple ways, relying on the use of technology and other dissemination strategies, while 
applying appropriate controls to protect student privacy. The Department must continue to 
invest in its information resources so that internal and external stakeholders can use the best 
available information to inform evidence-based decisions by states, districts, institutions of 
higher education, and students and parents. 


The Department continues to support states developing systems that will collect, manage, and 
appropriately report the valid, reliable data that are essential to achieving improvements across 
education, but there is much more work to do. In addition to supporting the development of the 
systems and structures that will provide education agencies across the nation with the data 
necessary to generate accurate information on student performance and other critical elements, 
the Department must continue to lead the national discussion of how these systems are best 
and most appropriately used to support students, improve instruction, address inequities in 
access and success, develop future teachers, and inform practice. 


Additionally, the Department must help ensure that states, districts, and institutions of higher 
education are using and sharing data in ways that meet the highest standards of data ethics and 
protect student privacy, including compliance with applicable privacy laws. The collection, 
storage, maintenance, and use of data must be responsible and must appropriately protect 
student privacy. Stewards and users of data must remember that these data describe real 
people and ensure that systems protect the rights of those people. Student privacy is now a 
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focal point across the country; over the past three years, a majority of states have enacted 
student privacy legislation, while also expanding data use. 


Better use of information, for policymakers, educators, institutional leaders, and students and 
parents, depends on information being accessible through reliable technology in formats that 
are helpful to various users. Data on students’ educational and related financial outcomes will 
enable accountability for institutions and help to support students in their educational and career 
pathways. Additionally, the Department strives to provide public access to its own data by 
sharing it in various formats appropriate for data novices, reporters, researchers, and 
developers. In addition, the What Works Clearinghouse (WWC) provides user-friendly 
syntheses of research evidence on various approaches and strategies in ways that are 
designed to be helpful to decision-makers. Taken together, these activities support the effort to 
help ensure that scarce dollars have their intended impact and empower states, districts, and 
institutions of higher education to become more dynamic learning organizations, especially in 
areas with little existing rigorous evidence. 


Goal 5 Discretionary Resources 


FY 2018 


FY 2017 


FY 2016 


$1,000 $1,500 $2,000 $2,500 
(Dollars in millions) 


WEN Colmm BU tXorg-)tColar-l AVM ea cele le-luitcw-lare mw Vel thi t(-S-Lemode]e) Lelat late me ey-1 MoM ala celaiil- Lares 
Metrics [Dollars in Millions] 


FY 2017 FY 2018 
FY 2016 Annualized President's 

POC Account Obj. Program Appropriation CR” Budget 
IES IES 5.3 National assessment 149 149 149 
IES IES 5.3 Regional educational laboratories 54 54 54 


72 All the programs listed are discretionary programs, as distinct from mandatory programs. These include both competitive and 
noncompetitive/formula programs. 

73 A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 
the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 
provided by the continuing resolution. 
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FY 2017 FY 2018 
FY 2016 Annualized President’s 
POC Account Obj. Program Appropriation CR? Budget 
IES IES 5.3 Research in special education 54 54 54 
IES IES 5.3 Research, development, and dissemination 195 195 195 
IES IES 5.1,5.2 | Statewide longitudinal data systems 35 34 34 
IES IES 5.3 Statistics 112 112 112 
Oll 1& 5.3 Education innovation and research” 120 120 370 
Subtotal 719 718 968 
Other Discretionary Programs/Activities 142 141 161 
TOTAL, GOAL 5 861 859 1,129 
POC = Principal Operating Component. 
CR = Continuing Resolution. 
NOTES: Many programs may have sub-activities that relate to other goals. Detail may not add to total due to rounding. 
74 This program was titled “Investing in Innovation” in 2016. 
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Goal 5: Details 


postsecondary data with 
workforce data 


INCREASE 


t 


28 OFY 2016 
Target 


mFY 2016 
Actual 
25 


Continuous Improvement Cc 
: urrent Current 
of the en Voce Year saa a Out-Year Targets 
Indicators of Success : Target | Results Trend Line 
Indicator Measurement (Actuals) 
= 
Direction 2014 2015 2016 2016 2016 Fea 2017 2018 
Missed” Exceeded’ 
5.1.A. Number of public 100 
data sats included in ED FY: 2013 | FY: 2014 FY: FY: 2016 | FY: 2016 MET 104 110 
Data Inventory and thus 55 66 2015 94 94 ae 80 
linked to Data.gov or 79 60 
ED.gov websites 40 
@ry 2016 20 
INCREASE Target 0 
mFY 2016 2014 2015 2016 
Actual 
5.1.B. Number of states FY: 2013 | FY:2014 FY: FY: 2016 | FY: 2016 MET 25 28 " 
linking K-12 and 12 20 2015 28 25 Pe 2 
24 20 


2014 2015 2016 


75 Missed target by <=1, or if percentage, <=1.3 percentage points. 


76 Surpassed target; not just met the target. If a diminishing target, the actual was below the reduction target set. 
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Continuous Improvement Current Current 
of the ano Year Year cae Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement : (Actuals) 
=] 
Direction 2014 2015 2016 2016 2016 al 2017 2018 
Missed’ Exceeded’6 
54. Number ofeiaics FY: 2013 FY: FY: FY: FY: MET NA NA ee 
linking K-12 with early 19 2014 2015 2016 2016 30 a 
childhood data” 26 32 35 29 Bs - 
40 
10 
INCREASE 30 29 F 
20 GFY 2016 2014 2015 2016 
Target 
ie mFY 2016 
0 Actual 
New Metric: Number of FY: 2012 18 28 32 NA NA NA 2678 35 NA 
states actively using data 7 


systems to support and 
inform improvements 


77 Retiring metric at conclusion of FY 2016. Please see appendix B for additional information pertaining to the metric’s retirement. The FY 2017 and 2018 targets were 32 and 33, 
respectively. If anew metric is being proposed, the new metric will be directly below the indicator measurement direction of the metric being retired. 
78 Currently finalizing approval from OMB to implement survey of all states on data system capabilities and uses. The appropriate time to revisit FY 2017 and 2018 targets will be in 
quarter 4 of FY 2017 once the Department has the initial data from that survey. 
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Continuous Improvement 


f; Current Current 
erie ano : Year Year cae Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement : (Actuals) 
= 
Direction 2014 2015 2016 2016 2016 Feel 2017 2018 
Missed’ Exceeded’6 
5.2.0. Average time to FY: 2013 FY: 2014 FY: FY: 2016 | FY: 2016 MET 6.709” 10.0 
aaa 1 201 i 7.2 
close “cases” (PTAC + 0 days 9 days ie 6.06 days days BBE 7 
FPCO) days ‘ 
8.0 6 
GFY 2016 2 
DECREASE sis Target 2 
6.0 8 
mFY 2016 2014 2015 2016 
Actual 
5.0 
4.0 
5.3.A. Percentage of select | FY: 2012 FY: 2014 FY: FY: 2016 | FY: 2016 MET eee 20.0% 30.0% 
new (noncontinuation) 6.5% 15.9% 2015 29.9% 18.0% nage ee 
competitive grant dollars 29.4% 30.0% 
that reward evidence Sake BFY 2016 25.0% a 
3 Target 20.0% 
15.0% 
20.0% mFY 2016 _ 
INCREASE nee — 
10.0% 0.0% 
ae 2014 ©2015 
0.0% 


7? Target was updated to reflect the goal of a 10% reduction from the prior year in the 2015 APR (FY 2017 target was an average of 6.48 days). However, the target has since been 
revised to be less aggressive due to internal staffing shifts, but still represents a 2 day improvement from the FY 2016 target. 
8° Metric is aligned to an Agency Priority Goal. This metric’s FY 2016 actual excludes Striving Readers. Even without that data, the metric’s target has been met. 
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Continuous Improvement 


0 


0 


‘ Current Current 
erie ano : Year Year cae Out-Year Targets 
Indicators of Success : Target Results Trend Line 
Indicator Measurement 5 (Actuals) 
SI 
Direction 2014 2015 2016 2016 2016 al 2017 2018 
Missed’> Exceeded’6 
FY: 2013 FY: 2014 FY: FY: 2016 FY: 2016 MET 40,892 46,892 
23,512 27,292 2015 47,573 35,692 
5.3.B. Number of peer- 36,197 EE nie 
reviewed, full-text sia iid 
resources in the Rano : 
Education Resources j a 30,000 
Information Center (ERIC) 40,000 20,000 
aoe 10,000 
30,000 92 : 
20,000 @FY 2016 2014 2015 2016 
Target 
INCREASE 10,000 aeoue 
t 0 Actual 
5.3.C. Number of visits to FY: 2015 NA FY: FY: 2016 FY: 2016 MET 2,164,536 | 2,380,989 
the What Works 1,822,000 2015 3,756,724 1,967,76 EB 
Clearinghouse (WWC) 1,822,0 0 Ameen 
website 00 3,000,000 
4,000,000 
3,75 pat eas 
3,000,000 ‘ 
INCREASE 4 mFY 2016 1,000,000 
2,000,000 Actual j 
1,96 2014 2015 2016 
1,000,000 7,76 
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Continuous Improvement 
of the U.S. Education 
System 
Indicators of Success 


Indicator Measurement 
Direction 


5.3.D. Number of 
completed project 
evaluations from grantees 
from select discretionary 
grant programs in a given 
fiscal year that meet What 
Works Clearinghouse 
(WWC) Evidence 
Standards*' 


INCREASE 


FY: 2015 
2 


Current Current 
Year Year cae Out-Year Targets 
Target Results Trend Line 
- (Actuals) 
2014 2015 2016 2016 2016 eer 2017 2018 
Missed’> Exceeded’6 
NA FY: FY: 2016 FY: 2016 MET 10° 30 
2015 20 10 
2 BEE 25 
20 
25 
a Gry 2016 - 
Target 10 
us mFY 2016 5 
Actual 


10 


2014 2015 2016 


51 Metric is aligned to an Agency Priority Goal (APG). 
82 The FY 2017 target has been revised to reflect a target of 10 versus 20. The APG statement notes that by 9/30/2017 there will 20 completed project evaluations. FY 2016's target 


was 10 and FY 2017’s target is also 10, equating to a target of 20 for the two-year APG. 
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Continuous Improvement 
of the U.S. Education 
System 
Indicators of Success 


Indicator Measurement 
Direction 


5.4.A. Percentage of 
schools in the country 
that have actual Internet 
bandwidth speeds of at 
least 100 Mbps 


INCREASE 


t 


FY: 2013 
20.0% 


Current Current 
p Year Year Actualite: Target Out-Year Targets 
2016 ‘ 
Target Results Trend Line 
: (Actuals) 
= 
2014 2015 2016 2016 2016 bis 2017 2018 
Missed’ Exceeded’6 
41.0% 55.0% 80.9% 70.0% MET 80.0% 90.0% 
aH 100.0% 
eee FY 2016 80.0% 
Target 60.0% 
mFY 2016 wean 
Actual 20.0% 
70.0 0.0% 
% 


2014 2015 2016 


Goal 5 FY 2016 Indicator Performance Summary 


9 Total Indicators 


10 


) 


(100.0%) 


m@ Met 
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NA = Not applicable. 

TBD = To be determined. 

Academic Year (AY) is a collegiate year spanning August—May; School Year (SY) spans August—July and is aligned with a P—12 school year; Fiscal Year (FY) corresponds to a federal 
fiscal year; Calendar Year (CY) spans January—December. 


Data Sources and Frequency of Collection: 

5.1.A. Data Strategy Team Data Inventory and the public ED Data Inventory at http://datainventory.ed.gov; quarterly 

5.1.B. State Longitudinal Data Systems (SLDS) grant monitoring (monthly updates from states, annual performance reports, final performance reports, and site visits); quarterly 

5.1.C. SLDS grant monitoring (monthly updates from states, annual performance reports, final performance reports, and site visits); quarterly 

5.2.A. Case Tracking System (CTS) Monthly Metric Reports; quarterly 

5.3.A. Forecast Report issued by the Office of the Chief Financial Officer (OCFO) and final Funding Reports from relevant programs; annually 

5.3.B. | Education Resources Information Center (ERIC); quarterly 

5.3.C. | What Works Clearinghouse (WWC) website analytics provided monthly by the WWC website contractor; quarterly 

5.3.D. Discretionary grant slate memoranda, discretionary grant financial forecasts and reports from OCFO, and the What Works Clearinghouse; quarterly 

5.4.A. Education Superhighway (for baseline), Consortium for School Networking (CoSN)/AASA (American Association of School Administrators today known as AASA, The 
School Superintendents Association) E-rate Infrastructure Survey; annually 


Note on performance metrics and targets: These metrics were established as a part of the FY 2014-18 Strategic Plan. Metrics may be updated or revised to 
reflect awareness of more accurate data or clarifications. Such updates or revisions are identified in footnotes. 
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Analysis and Next Steps by Objective 


Objective 5.1: Data Systems and Transparency. Facilitate the development of 
interoperable longitudinal data systems for early learning through employment to enable 
data-driven, transparent decision making by increasing access to timely, reliable, and 
high-value data. 


FY 2016 Implementation Strategy 


The primary implementation strategies regarding data systems and transparency for FY 2016 
were centered on three goals: 


e ensure a successful first year for the 16 SLDS grantees from the FY 2015 round, 
e connect internal transparency efforts to the new InformED initiative, and 


e improve the tools and support services available to the public for adoption of the CEDS. 


With the new SLDS grantees, the Department looked at grant areas that created problems in 
previous grants and aligned support as appropriate when setting up grant monitoring plans for 
this round of grantees. In addition to establishing clear grant implementation plans and 
monitoring schedules with each FY 2015 grantee, the SLDS team delivered 15 topical webinars 
and released 9 new publications on traditionally challenging topics, including data system 
sustainability, effectively linking education and workforce records, and successfully supporting a 
research agenda. These developed resources are available for all states, not just grantee 
states, through the SLDS program website. 


Within the Department, the InformED initiative was launched during FY 2016. InformED seeks 
to develop a world-class open data infrastructure at the Department, focusing on improved data 
releases and internal data dissemination procedures. One of the important FY 2016 
accomplishments was to improve the Department’s data landing page, including enabling 
visitors to search data resources by topic and keywords. By the end of the year, the InformED 
activities continued to expand to encompass a study of aligning key words in the Data Inventory 
with the organizational structure used on the new landing page, thereby aligning existing Goal 5 
metrics with InformED activities. 


FY 2016 began with the installation of a new support team for the CEDS. The team analyzed 
public feedback and website usage to inform their decision to place a high priority on 
reorganizing the tools and resources on the CEDS webpage. This strategy resulted in the 
deployment of a Mapping Toolkit on the website before the end of the fiscal year. 


FY 2016 Barriers to Success 
There are three key barriers that could affect progress on this strategic objective: 


1) The Department may not have the human capital and financial resources needed to support 
the information technology (IT) infrastructure and procedural changes required to continue to 
advance its open data and transparency efforts. 


2) The lower number of active SLDS grantees (as FY 2012 grants close out) could lead to 
program data not accurately representing the state of the nation as a whole. 
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3) Focus on other online resources could result in less traffic to education.data.gov, which could 
make it more difficult to track usage statistics and improve the sites accordingly and may require 
redefining success in terms of web traffic. 


Key Milestones and Future Actions 
The Department reached a number of key milestones, including: 


e launching CEDS Mapping Toolkit; 
e establishing grant implementation plans for all 16 FY 2015 SLDS grantees; 


e deploying resources and technical assistance to FY 2015 SLDS grantees, and making 
additional opportunities available to all states; and 


e adding information on 16 public datasets to the ED Data Inventory, and through the 
Department’s data.json file to the repository at data.gov. 


In FY 2017, the Department engaged a network of state Chief Information Officers (ClOs) 
organized by Chief State School Officers (CSSOs) at their fall meeting about explicit actions 


they would be willing to take in support of CEDS. There was consensus among the members in 
attendance at the fall meeting that strong messaging to establish CEDS as the standard listing 
of elements, definitions, and relationships upon which they rely was needed. In addition to that 
messaging, the network of ClOs of CSSOs is drafting a set of “action steps” that member states 
can choose to implement. 


The SLDS program team spent much of FY 2016 preparing a new data collection proposal to 
enable gathering information on data system and transparency capability from all states, not just 
grantee states. 


Objective 5.2: Privacy. Provide all education stakeholders, from early childhood to adult 
learning, with technical assistance and guidance to help them protect student privacy 
while effectively managing and using student information. 


FY 2016 Implementation Strategy 


The past several years have seen significant activity on student privacy issues. News stories 
abound about data collection in schools from emerging technologies, and the majority of states 
have passed student privacy legislation. Various federal student privacy statutes were 
introduced as well, though none of them were enacted. Resources devoted to student privacy 
increased in FY 2016, with five new full-time equivalents being added to these operations, and 
with the introduction of privacy “Fellows” in the Office of the Chief Privacy Officer. These new 
resources have enabled the Department to continue to meet the growing cry for technical 
assistance on privacy matters. The addition of these new resources, and a desire to focus on 
policy development and enforcement, prompted the Department to reorganize its student 
privacy functions, effective January 8, 2017. Two divisions have been created under the Chief 
Privacy Officer. The Family Policy Compliance Office (FPCO) will continue with its traditional 
function of investigating and responding to complaints from parents and eligible students, as 
well as providing technical assistance to school officials related to those complaints. A new 
division, the Student Privacy Policy and Assistance Division (SPPAD), will lead efforts to 
develop Departmental policy and coordinate technical assistance. 


The Department continues to refine the efficiency of its technical assistance delivery through 
use of metric management, relying on a case tracking system that manages workload and 
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content for both contractor and Department staff. These gains were further expanded during 
implementation of the restructured Privacy and Technical Assistance Center (PTAC) contract 
during the first quarter of the fiscal year, which increased contractor resource utilization by 
housing the helpdesk function within headquarters. 


Throughout the fiscal year, the Department made progress reducing turnaround time for cases 
and providing targeted technical assistance. Reducing the backlog of complaints and inquiries 
required devising and implementing a new approach in assigning and tracking cases. In 
addition, the Department completed a comprehensive review of the data in the tracking system, 
and worked to address a data entry delay uncovered during third-quarter reporting regarding 
average age of open “correspondence and complaints” and quality control for closing cases. 


FY 2016 Barriers to Success 
The Department faces two primary barriers to success: 


e While the Department made progress on policy development in FY 2016, significant 
work is still required to answer emerging and longstanding policy questions on privacy 
topics related to video recording, e-mails, and permissible use. 


e The Department’s student privacy caseload continues to increase, as does the complaint 
backlog in FPCO. Process improvements and expanded proactive technical assistance 
helped to mitigate the impact of this growth, but the sharp increase in new complaints 
resulted in a modest increase in the complaint backlog of 21 percent over the same 
timeframe. While the new resources and reorganization should help, limited resources 
present a challenge to support student privacy technical assistance and enforcement 
activities. 


Key Milestones and Future Actions 


During FY 2016 and in prior fiscal years, the Department demonstrated expected progress on 
the metrics related to technical assistance delivery. Turnaround time for cases averaged only 
6.06 days, exceeding the goal of less than 7.2 days by more than a day. Proactive technical 
assistance goals were achieved through site visits, presentations, webinars, and regional 
meetings. To focus efforts on improving response time, and as noted in the FY 2015 APR, the 
metric regarding “average time to close correspondence and complaints” was retired and 
replaced with the “average age of correspondence and complaints.” 


In achieving the metric goals, the Department accomplished several additional milestones. In 
FY 2015, the Department responded to public and Congressional criticism over the privacy of 
students’ medical treatment records in the wake of a recent sexual assault case by issuing a 
draft Dear Colleague Letter to obtain public input on the protection of student privacy in campus 
medical records. After extensive collaboration not only across the Department, but also with 
partners in HHS, in FY 2016, the Department released and was publicly commended for the 


final version of this important Dear Colleague Letter. 


Another significant accomplishment was the Department's collaboration with DOL in providing 


needed guidance to state agencies, educational agencies and institutions, and service providers 
on performance reporting and evaluation requirements under WIOA. 


In addition to these formal guidance documents, the Department also offered technical 
assistance through a variety of short, informative videos targeting both school personnel and 
parents. 
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Objective 5.3: Research, Evaluation, and Use of Evidence. /nvest in research and 
evaluation that builds evidence for education improvement; communicate findings 
effectively; and drive the use of evidence in decision making by internal and external 
stakeholders. 


FY 2016 Implementation Strategy 


In FY 2016, the Evidence Planning Group (EPG), which consists of representatives from Oll, 
IES, and OPEPD, followed a similar implementation strategy to that of past years. Prior to the 
start of FY 2016, EPG met with various programs to discuss whether it would be appropriate to 
move toward an evidence-based model in their competitions. In the first quarter of FY 2016, 
programs worked with EPG to finalize their plans for using evidence once funding levels 
became certain. In addition, EPG began to review the ESSA carefully to determine which 
programs would be the best candidates for evidence, as well as how the Department’s current 
evidence definitions are aligned with the ESSA’s “evidence-based” definition, which appears in 
several programs. In the fourth quarter of FY 2016, the Department released guidance for states 
and districts that suggests steps for effective decision-making using evidence and recommends 
criteria and considerations for each of the four levels of evidence in the ESSA. Additionally, the 
Department's Office of Educational Technology (OET), in partnership with IES, contracted 
Mathematica Policy Research and SRI International to build an online platform called the Ed 
Tech Rapid Cycle Evaluation (RCE) Coach to support school and district leaders to collect more 
evidence when making decisions about educational technologies. The need for evidence-based 
decision-making found in the ESSA prompted this work, and the platform is now available and 
free for educators to use. 


The Department exceeded the FY 2016 target for number of peer-reviewed, full-text resources 
in the Education Resources Information Center (ERIC). In the first quarter, ERIC completed a 
new source selection round and approved new sources to be in ERIC. In addition, IES engaged 
the i3 program to encourage its grantees to submit the studies from their project evaluations to 
ERIC, achieving two important goals for the i3 program: (1) ensuring that all studies from the i3 
program are publicly available and (2) ensuring that studies are made available for WWC review 
in a systematic way. 


In FY 2016, the Department exceeded the planned target for number of reviewed studies in the 
WWC database. In the first quarter, to continue to increase the number of visitors to the WWC 
website, the WWC continued to expand the database of reviewed studies through reviews 
conducted for WWC products (e.g., intervention reports, practice guides, single study reviews, 
and quick reviews). Studies submitted as part of a grant application for the Department's 
evidence-based grant competitions were also a source of evidence reviewed by the WWC. 
During the first quarter, the WWC reviewed studies for grant competitions and publicized the 
use of its study findings dataset, which is a resource that reports all available data for studies 
that meet standards and either have a WWC report or were reviewed for a grant competition. In 
addition, in the fourth quarter of FY 2016, the WWC released a redesigned “Find What Works” 
tool that allows users to search for studies by topic area, such as math or science, to find 
studies where there is evidence of positive effects. The “Students Like Yours” feature of this tool 
also allows users to specify characteristics of their students to better identify what research has 
been conducted on similar populations. This resource continues to be widely used among the 
research community. To ensure success for this metric, IES continually collaborated with staff 
from Oll, the Office of Postsecondary Education (OPE), and OELA to coordinate evidence 
reviews for evidence-based competitions. 
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In FY 2016, the Department exceeded the target for number of completed project evaluations 
from grantees of select discretionary grant programs that meet WWC evidence standards. The 
main implementation strategy for this indicator involved checking the WWC database of 
reviewed studies to determine whether any new studies from Department-funded competitive 
grants were in the database and met WWC standards. Bolstering performance for this metric is 
the Department’s understanding that effective technical assistance is necessary to ensure 
grantees tasked with conducting rigorous evaluations of their projects stay on track. 


FY 2016 Barriers to Success 


The EPG continues to consider whether the current approach to this work is sustainable. Based 
on lessons learned, the Department has determined that focus must be strategic so that the use 
of evidence is in select programs rather than continuing to scale at the current rate. With limited 
resources, it is crucial that the Department focus on high-quality work in programs where using 
evidence is most likely to be impactful, as opposed to putting evidence priorities in every 
competitive program without strong fidelity of implementation. 


It is important to note that these efforts are also complemented by the work of the Department's 
Comprehensive Centers and Regional Educational Laboratories, which are designed to provide 
high-quality resources to the field. 


While the Department surpassed its target for the number of studies conducted as part of a 
discretionary grant-funded project that are determined to meet WWC evidence standards in 
FY 2016, this work may not be sustainable. One lesson from the i3 program is that, even when 
employing very sophisticated evaluators, substantial technical assistance from the Department 
is essential in order to keep the project evaluations on track to meet rigorous standards. While 
most discretionary grant programs do not have the resource flexibility to focus on rigorous 
evaluation standards, EPG has worked to create a contracting option for programs that need 
help with evaluation expertise, with a few programs entering into that contract for FY 2016. 


Key Milestones and Future Actions 


Looking forward to FY 2017, the Department continues to consider ways to streamline and 
improve upon its evidence review process for discretionary grant competitions. 


To increase the number of visits to the WWC website in FY 2017, the Department will continue 
to expand social media presence, point competitive grant applicants to the WWC website, and 
produce products like intervention reports, quick reviews, single study reviews, and practice 
guides. 


Objective 5.4: Technology and Innovation. Accelerate the development and broad 
adoption of new, effective programs, processes, and strategies, including education 
technology. 


FY 2016 Implementation Strategy 


The Department’s many successes during FY 2016 include expanded technical support and 
assistance to support state and district leaders across the country working to improve teaching 
and learning through the effective use of technology. Over 25 states and the District of 
Columbia have joined the effort and committed to supporting district leadership teams in 
planning for digital learning. In addition, 3,000 superintendents from across the country have 
committed to foster and lead a culture of digital learning in their districts by signing the Future 
Ready District Pledge. The Department, in partnership with the Alliance for Excellent Education 
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and with support from a coalition of over 50 national and regional partners, supports district 
leaders with specific tools and guidance to plan and implement personalized, research-based 


digital learning strategies in order to prepare students for success in college, career, and 
citizenship. 


To support the work of the superintendents, OET continued to collect examples of best 
practices for connecting schools, providing devices, and preparing teachers to use technology 
effectively. These regularly updated resources were posted on the Department’s website, blog, 
and YouTube channel and shared via Twitter and Facebook. 


In October 2015, OET launched #GoOpen, a national movement that supports states, school 
districts, and educators transitioning to openly licensed educational materials to transform 
teaching and learning. Openly licensed educational resources have enormous potential to 
increase access to high-quality educational opportunities in the United States when they are 
accessible via high-speed broadband. 


#GoOpen was launched on October 29, 2015, at the Open Education Symposium, an event that 
brought together district leaders, state leaders, nonprofits, foundations, and private sector 
companies. As of September 30, 2016, 76 districts committed to transitioning to the use of 
openly licensed educational resources to replace traditional, static instructional materials, and 
16 states committed to providing guidance and leadership for districts making this transition, as 
well as developing a statewide repository to search and discover resources. 


FY 2016 Barriers to Success 


Although much progress has been made in connecting schools to high speed broadband, future 
roadblocks to progress include access to needed fiber optic cable, especially in rural areas; 
affordability of broadband, especially in smaller and rural districts that are not eligible for bulk 
pricing discounts; and fully utilizing E-rate funds, since some districts still struggle to provide a 
percentage match to every E-rate dollar they receive. Several challenges remain in meeting the 
goals of this objective, including the need to educate the public about privacy and data security 
(leading to setbacks in the ability to use data to create personalized learning systems), difficulty 
measuring effectiveness without a robust evaluation program, and difficulty showing impact 
without data collection. 


Limited resources may prevent OET from meeting its legislative mandate to provide technical 
assistance to states, districts, and programs across the Department and the federal 
government. This technical assistance has included research and evaluation, updating critical 
guidance documents, and providing ongoing partnership support to organizations to leverage 
technology to improve teaching and learning and in support of states and districts working to 
increase connectivity for students. 


Key Milestones and Future Actions 


Strategies for reaching this goal have included updating federal government policies and 
guidance, encouraging significant private sector commitments, and engaging in national 
outreach efforts to states, districts, technology providers, and nonprofit organizations. 
Collaboration within the Department, with other government agencies, and the education 
community as a whole is also key to success, as is developing sustainable, scalable solutions 
for using data and evidence in decision-making. Some strategies, especially around support for 
teachers adopting innovative approaches, evaluating effectiveness of technology-based 
approaches, and establishing best practices for maintaining student privacy, are dependent on 
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funding and staffing. The Department is working to identify ways to make its data more 
accessible and actionable for the public. 


In the coming year, the Future Ready District Infrastructure Guide will be updated with an 
expanded connectivity section. OET will continue to collect examples of best practices for 
connecting schools, providing devices, and preparing teachers to use technology effectively and 
share them via its social media channels. 


Work will continue on the Ed Tech RCE Coach tool during FY 2017. With the implementation of 
ESSA, the need for evidence-based decision-making has increased significantly. The goal is to 
fundamentally change the procurement and implementation process to include evidence-based 
decision-making throughout. 


During FY 2017, OET will collect new examples from the field to add to the National Educational 
Technology Plan. This is based on the office’s commitment to refresh the plan more frequently 
than the previous five-year cycle in order to better respond to the needs of the field. 
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Goal 6. U.S. Department of Education Capacity: 


Improve the organizational capacities of the Department to 
implement the Strategic Plan. 


Goal Leader: Assistant Secretary, Office of Management (OM) 


Objective 6.1: Effective Workforce. Continue to build a skilled, diverse, and engaged 
workforce within the Department. 


Objective 6.2: Risk Management. Improve the Department’s program efficacy through 
comprehensive risk management, and grant and contract monitoring. 


Objective 6.3: Implementation and Support. Build Department capacity and systems to 
support states’ and other grantees’ implementation of reforms that result in improved outcomes, 
and keep the public informed of promising practices and new reform initiatives. 


Objective 6.4: Productivity and Performance Management. Improve workforce productivity 
through information technology enhancements, telework expansion efforts, more effective 
process performance management systems, and state-of-the-art leadership and knowledge 
management practices. 


ed0 le) (om ={:1ai=Vi1! 


The Department continues to focus on acquiring and developing its workforce through human 
capital management; increasing diversity and inclusion and improving employee engagement; 
rethinking how it monitors and intervenes with high-risk grantees and contractors; enhancing 
workforce productivity through IT; safeguarding its assets and stakeholders from cybersecurity 
threats; continuing to improve and integrate effective performance management; and 
transforming the way the Department interacts with states, districts, institutions of higher 
education, and other grantees and stakeholders. These efforts aim to improve performance 
results, increase stakeholder collaboration, and lead to greater employee engagement. 


The Department continues to build Department capacity and systems to support states’ and 
other grantees’ implementation of reforms that result in improved outcomes, and keep the public 
informed of promising practices and new reform initiatives. By consolidating processes, the 
Department has been able to more effectively customize its outreach to individual states and 
model the critical partnerships states should have with their respective districts. 


Beyond building Department capacity to support states and other grantees, throughout 

FY 2016, the Department provided strategic training courses to strengthen leadership and 
knowledge management throughout the Department, with a special emphasis on ensuring 
managers and supervisors have the essentials they needed to effectively manage and lead. The 
Department also recognized the important role that technology and facilities contribute to 
supporting productivity. As a result, the Department continued to build on the success of its ED 
Space Modernization plan, including the deployment of critical IT to support telework and 
leverage wireless connectivity. 
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Goal 6 Discretionary Resources 


FY 2018 


FY 2017 


FY 2016 


$1,000 $1,500 $2,000 $2,500 
(Dollars in millions) 


WN EKoy am BJ CXevg-) tColar-lavm manele le-liitcm- lave M-Vel (aid (-t-haaeode] o) Lelailale MCLor-] MoM m-vaeleuir-larer-) 
Metrics [Dollars in Millions] 


FY 2017 FY 2018 
FY 2016 Annualized President’s 
POC Account Obj. _ Program Appropriation CR™ Budget 
OIG OIG Office of Inspector General 59 59 61 
Program Administration: Building 
OMIPE: | eMien modernization 1 -- -- 
Program Administration: Salaries and 
PRE: (PMIEE expenses 431 431 438 
TOTAL, GOAL 6 491 490 499 


POC = Principal Operating Component. 
CR = Continuing Resolution. 
NOTES: Many programs may have sub-activities that relate to other goals. Detail may not add to total due to rounding. 


83 All the programs listed are discretionary programs, as distinct from mandatory programs. These include both competitive and 
noncompetitive/formula programs. 

84 A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 
the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 
provided by the continuing resolution. 
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(Coy: | mofam BY=yr-11 (sy 


U.S. Department of Current Current Lee! 
Education Year Year ss arget Out-Year Targets 
Indicators of Success Baccling Target Results Trend Line 
Indicator = (Actuals) 
Measurement 2015 2016 2016 2016 0 ea 6 2017 2018 
Direction Missed Exceeded 
6.1.A. Percent of FY: NA FY: 2015 FY 2016: FY: 2016 MET Fa 51.2% 53.7% 
selections made per job 2015 46.4% 70.0% 48.7% ates 
opportunity 46.4% GFY 2016 om 
announcement (JOA) 60.0% | em | Tareet 60.0% ZL 
cid mi FY 2016 
INCREASE 40.0% Rees 40.0% 
48. 20.0% 
20.0% 7% 
0.0% 
0.0% 2014 2015 2016 
6.1.B. EVS Employee FY: 2012 FY: 2014 FY: 2015 FY 2016: FY: 2016 NOT 73-078 71. 72.0% 73.0% iii 
Engagement Index 64.7% 67.0% 68.0% 67.0% 71.0% MET 0% ERFY 2016 we 
70.0% Target ae en 
66.0% 
INCREASE mFY 2016 64.0% 
OY 67. 
65.0% a Actual eae 
60.0% 
60.0% 2014 2015 2016 
6.1.C. Time to hire FY: 2013 FY: 2014 FY: 2015 FY: 2016 FY: 2016 MET 70.0% 71.0% inane 
65.0% 85.0% 67.6% 79.1% 69.0% Fee sean 
80.0% Ory 2016 60.0% N\A 
INCREASE 75.0% + [om Tareet 40.0% 
mFY 2016 20.0% 
70.0% Actual 0.0% 
2014 2015 2016 
65.0% a 
60.0% 


85 Missed target by <=1, or if percentage, <=1.3 percentage points. 


86 Surpassed target; not just met the target. If a diminishing target, the actual was below the reduction target set. 
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U.S. Department of Current Current La 
Education Actuals Year Year oe arget Out-Year Targets 
Indicators of Success Bence Target Results Trend Line 
Indicator SS (Actuals) 
Measurement 2014 2015 2016 2016 2016 5 ta a 2017 2018 
Direction Missed Exceeded 
6.1.D. Effective FY: 2012 FY: 2014 FY: 2015 FY: 2016 FY: 2016 NOT RSS 52.0% 53.0% 
Communication Index 48.0% 50.0% 51.0% 50.0% 51.0% MET MN“ ef = 
51.0% I Oy, 50.0% + pew 
49.0% 50 ies 
47.0% ene 
eee OFY 2016 
INCREASE ee Target oe 
35.0% 
coup mFY 2016 2014 2015 2016 
39.0% Actual 
37.0% 
35.0% 
6.2.A. Percentage of FY: 2012 FY: 2014 FY: 2015 FY: 2016 FY: 2016 MET Ree NA NA ae 
A-133 Single Audits 57.0% 37.0% 20.0% 10.0% 37.0% ae = 
Overdue for resolution®’ 30.0% 
37. i @FY 2016 
a Oe aes 20.0% 
DECREASE 20.0% a eehe 10.0% 
Actual 0.0% 
10.0% 2014 2015 2016 
10. 
0.0% +d 


87 Retiring metric at conclusion of FY 2016. Please see appendix B for additional information pertaining to the metric’s retirement. The FY 2017 and 2018 targets were 31.0% and 
25.0%, respectively. If a new metric is being proposed, the new metric will be directly below the indicator measurement direction of the metric being retired. 
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U.S. Department of Current Current LA 
Education Actuals Year Year oe arget Out-Year Targets 
Indicators of Success Bence Target Results Trend Line 
Indicator SS (Actuals) 
Measurement 2016 2016 HEHE 2017 2018 

Direction Missed® | Exceeded® 

New Metric: Percentage FY: 2014— NA NA 86.1% NA NA NA 86.1% 86.6% NA 

of Department Grant 2016 

Recipients without any Average 

Single Audit Findings 85.4%% 

6.2.B. Compliance rate of FY: 2013 FY: 2014 FY; 2015 FY: 2016 FY: 2016 NOT 100.0% 100.0% anoine 

contractor evaluation 85.0% 97.0% 98.0% 95.8% 100.0% MET Huns : 


performance reports 


INCREASE 


t 


95.0% 


90.0% 


85.0% 


@FY 2016 
Target 


mFY 2016 
Actual 


“sy, 


95.0% 
90.0% 


85.0% 
201420152016 


88 The baseline data is based on an average of Department grantees with no single audit findings over the past three fiscal years, 2014-16. 
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U.S. Department of 


Education 


Indicators of Success 


Indicator 


Measurement 


Direction 


6.3.A. Overall average 


impact score of the 


Department’s technical 
assistance in helping 
build State capacity to 
implement education 


reforms®® 


INCREASE 


t 


8.00 


7.00 GFY 2016 


Target 
6.00 sree 


MFY 2016 
Actual 


5.00 


4.00 


Current Current 
Actuals Year Year Betuabte-Target Out-Year Targets 
2016 ; 

: Target Results Trend Line 
Baseline . (Actuals) 

2015 2016 2016 2016 HEE 2017 2018 

Missed® | Exceeded® 
FY: 2015 NA FY: 2015 FY: 2016 FY: 2016 NOT = 7.75 8.00 
778 7.73 7.58 8.00 MET ~ ras 


7.0 


6.5 
2014 2015 2016 


8° Metric has been revised from tracking the “percentage of states” to tracking the “overall average impact score” of the states that rate the Department’s technical assistance via the 
Grantee Satisfaction Survey, which is a more meaningful metric for the Department. The baseline and subsequent data points have been revised from the 2015 APR to reflect the 


change in the metric. 
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U.S. Department of Current Current red 
Education Actuals Year Year gigs arget Out-Year Targets 
Indicators of Success Baesiing Target Results Trend Line 
Indicator SS (Actuals) 
Measurement 2014 2015 2016 2016 2016 is Fae ie 2017 2018 
Direction Missed Exceeded 
6.4.A. Number of ED IT FY: 2012 FY: 2014 FY: 2015 FY: 2016 FY: 2016 MET retst| 291° 277"! ipa 
security incidents 756 445 580 291 551 
600 600 
DECREASE 00 THis gry 2016 = “A 
400 Target 200 
300 0 
x00 mFY 2016 2014 2015 2016 
Actual 
100 
0 
6.4.B. EVS Results- FY: 2012 FY; 2014 FY: 2015 FY: 2016 FY: 2016 NOT SS 58.0% 59.0% 
Oriented Performance 53.0% 56.0% 57.0% 56.0% 57.0% MET \S ee sail 
Culture Index pp 0% 55.0% tm 
55.0% 
soo Wee @ry 2016 50.0% 
INCREASE ~ | eet 
t o> 076) mee — 2014 2015 2016 


°° FY 2017 target was reduced significantly to aim at a continual decrease in incidents by more than the 5 percent reduction from the initially proposed FY 2016 target in the 2015 APR. 
*' Reduction of 5 percent from previous year’s actual to align with a more aggressive approach to reducing security incidents. 
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U.S. Department of Current Current red 
Education Actuals Year Year iat arget Out-Year Targets 
Indicators of Success Baesiing Target Results Trend Line 
Indicator 3 (Actuals) 
Measurement 2014 2015 2016 2016 2016 a a a 2017 2018 
Direction Missed Exceeded 
6.4.C. EVS Leadership FY:2012 | FY:2014 | FY:2015 | FY:2016 | FY:2016 | NOT | 5%% 63. 64.0% 65.0% | oo. 
and Knowledge 60.0% 61.0% 62.0% 61.0% 63.0% MET al” 
Management Index re a pos 
”” | PRE ry 2016 wi 
INCREASE sox |e TELE 55.0% 
mFY 2016 a 
Actual ; 2014 2015 2016 
50.0% 
6.4.D. Total usable FY: 2012 FY: 2014 FY: 2015 FY: 2016 FY: 2016 MET esa TBD TBD 
square footage 1,563,641 | 1,533,239 | 1,530,864 | 1,367,000 | 1,459,937 peel 
1,500,000 1,400,000 
1,300,000 
1,450,000 Ory ,200, 
DECREASE ee anne 
1,400,000 Target re 
1,000,000 
2014 2015 2016 
1,350,000 ary 
2016 
Actual 
1,300,000 
6.4.E. Rent cost FY: 2014 FY: 2014 FY: 2015 FY: 2016 FY: 2016 MET Peace $74,470,439 TBD Pern 
$74.3M $74.1M $72.7M | 72,149,82 | $80.3M sno 000 ae secre 
8 9,93 72,500,000 ~ 
1,450,000 7 70,500,000 
DECREASE 68,500,000 
e100 000 GFY 2016 66,500,000 
voce a or 
1,300,000 Od FY 2016 "99142015 2016 
0 Actual 
1,250,000 
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Goal 6 FY 2016 Indicator Performance Summary 


14 
12 Total Indicators 


12 


10 


6 (50.0%) 


6 (50.0%) 


mw Met m@ Not Met 


NA = Not applicable. 

TBD = To be determined. 

Academic Year (AY) is a collegiate year spanning August—May; School Year (SY) spans August—July and is aligned with a P—12 school year; Fiscal Year (FY) corresponds to a federal 
fiscal year; Calendar Year (CY) spans January-December. 


Data Sources and Frequency of Collection: 

6.1.A. EDHires (Monster's electronic hiring management system); annually 

6.1.B. | Office of Personnel Management (OPM) Federal Employee Viewpoint Survey (FEVS); annually 

6.1.C. Federal Personnel/Payroll System (FPPS) Datamart; annually 

6.1.D. OPM FEVS; annually 

6.2.A. Office of the Chief Financial Officer's (OCFO) Audit Accountability & Resolution Tracking System (AARTS); annually 
6.2.B. Past Performance Information Retrieval System (PPIRS) www.ppirs.gov “PPIRS Compliance Report”; annually 
6.3.A. Annual Grantee Satisfaction Survey; annually 

6.4.A. Cyber Security Assessment and Management (CSAM) and RSA Security Operations management (SecOps); quarterly 
6.4.B. OPM FEVS; annually 

6.4.C.. OPM FEVS; annually 

6.4.D. Department's Master Space Management Plan; annually 

6.4.E. Department's Master Space Management Plan; quarterly 


Note on performance metrics and targets: These metrics were established as a part of the FY 2014-18 Strategic Plan. Metrics may be updated or revised to 
reflect awareness of more accurate data or clarifications. Such updates or revisions are identified in footnotes. 
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Analysis and Next Steps by Objective 


Objective 6.1: Effective Workforce 


FY 2016 Implementation Strategy 


In FY 2016, the Department continued to improve its time-to-hire performance, fill mission- 
critical positions, and offer viable options to the competitive process while ensuring hiring 
managers continue to receive a high caliber of applicants from which to select. These 
successes continue to be relayed during Supervisor Training 101, at collaboration meetings with 
hiring managers and HR Specialists, and in meetings with the Department’s Senior Leaders. 


The Department’s enhanced engagement activities resulted in the Department exceeding the 
government average Federal Employee Viewpoint Survey (FEVS) response rate by 

30.4 percentage points. The Department’s 2016 response rate increased 3.5 points to 

76.2 percent. Nine offices met or exceeded the Department’s 2016 response rate goal of 

80 percent. Fourteen offices exceeded their prior year participation rate; seven exceeded by 
double digits. The Department’s unwavering and focused championing of engagement has 
resulted in the Department achieving a 1 percent increase in the Employee Engagement Index 
annually since 2012. 


Diversity and inclusion has been empirically and positively associated with greater talent 
utilization, better employee retention, increased innovation, and higher performance. The 
Department continues to build capacity at all levels of the Department, working through the 
Department’s Diversity and Inclusion Council, the Diversity Change Agent Program, and various 
training opportunities. The Department has designed a “Diversity and Inclusion Dashboard” for 
internal use that serves as a tool to provide senior leaders with demographic diversity data 
about hiring, attrition, retention, and a host of other data-driven accountability measures to 
assist in diversity planning. The Department continues to participate in the governmentwide 
Federal Diversity in Government Council. 


FY 2016 Barriers to Success 


There are challenges to educating managers on the numerous hiring flexibilities of the 
recruitment process and engaging subject matter experts (SMEs) where it would be most 
beneficial. Some hiring managers found reworking recruitment packages to include 
strengthening specialized experience statements or reworking self-assessment questions to be 
a challenge. The Department mitigates this risk by building partnerships up front, utilizing other 
avenues to hire, and providing consistent briefings from top to bottom. 


The Department’s employee engagement initiative relies heavily on principal office prioritization 
and implementation. The Office of Management (OM) offered and will continue to offer training 
courses, access to expert consultants, and senior-level engagement meetings to assist principal 
offices in successfully implementing employee engagement programs and activities. OM also 
communicated to principal offices about the support services available to the principal offices to 
assist them in the engagement planning and implementation process. 


Key Milestones and Future Actions 


The Department has entered into an agreement with Monster Government Solutions to utilize 
the assessment tool and position classification modules within its hiring system. This agreement 
expands the Department’s ability to standardize position descriptions and provide more support 
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in the efforts to shorten timeframes. In addition, the Department will continue to promote the 
effective use of noncompetitive hiring authorities and advocate HR Specialists directly 
partnering with hiring officials at the onset of the process. 


In the area of employee engagement, OM will continue outreach to principal offices and work 
through the Monthly Operations Forum to shape annual engagement focus areas, raise 
awareness of best practices and information sharing, incorporate engagement best practices 
into day-to-day operations, and strive to increase the Department’s Employee Engagement 
Index. 


Currently, the Department plans to continue hosting regularly scheduled Diversity and Inclusion 
Council meetings, participating in the governmentwide Diversity and Inclusion Council, and 
providing diversity and inclusion training opportunities. 


Objective 6.2: Risk Management 
FY 2016 Implementation Strategy 


Through risk management, the Department identifies, sets priorities, and takes action on 
challenges affecting the successful use of grant, loan, and contract funds, in order to forward 
the achievement of its mission and strategic goals. During FY 2016, the Department’s risk 
management work included improving the audit resolution process, conducting grantee risk 
assessments, increasing oversight and technical assistance with targeted grantees, recording 
past performance information on contractors, and monitoring grants and contracts. 


This year, the Department improved its use of audit data to identify grantee management 
challenges. Our continued improvement in the audit resolution and closure process enables the 
Department to provide feedback and technical assistance to audited grantees that helps them 
improve the management and outcomes of their grant-funded activities. 


During FY 2016, the Department conducted a preaward risk review of all organizations slated 
for new awards or continuation awards from competitive grant programs. This process helped 
identify grantees that had not completed audits, as required for all grantees expending $750,000 
or more during the fiscal year, and resulted in many organizations completing their audits. The 
number of grantees the Department identified as missing audits declined by two-thirds between 
2012 and 2015. In addition, the Department’s program officers provided targeted oversight of 
and technical assistance to grantees to address issues identified during the preaward risk 
reviews. The Department formula grant programs reviewed the financial management and 
performance information of grantees and used this information to guide technical assistance to 
the field, as well as monitoring and oversight of specific grantees. 


Contract monitoring was improved this year by partnership between the contracts office and 
program staff to ensure that both the contractor fulfillment of requirements and successful 
outcomes and deliverables were achieved. Program offices conducted ongoing monitoring of 
grants, targeting programs and grantees that pose the greatest risk to program success. 


FY 2016 Barriers to Success 


Although the Department did not encounter barriers to success in risk management, there are 
ongoing challenges that must be addressed on a regular basis. These include the resources 
available for monitoring and the agency’s dependence on direct grant recipients to monitor the 
ultimate recipients, who spend the grant funds. 
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The staff levels in Department program offices, FSA, the contracts office, and other Department 
offices largely determine the limits of monitoring and oversight activities. To address this 
challenge, the Department continues to explore ways to make the process more efficient, such 
as targeting oversight based on risk, automating the analysis of audit and past performance 
information, and using telecommunication and web-based technology to communicate with 
grantees. 


Because most Department funds flow through direct grant recipients to the agencies and 
individuals who ultimately use the funds, good oversight of Department funds depends on the 
“pass through agency” that distributes funds to the ultimate recipient agencies and beneficiaries. 
State agencies that sub-award Department grants to local agencies are crucial participants in 
grant oversight. Control over student aid funds is dependent on participating institutions. Most of 
these partners in administering Department programs are also challenged to find sufficient staff 
resources to conduct rigorous oversight of grant programs. 


Key Milestones and Future Actions 


In 2016, the Department launched Enterprise Risk Management (ERM) to meet the 
requirements of OMB Circular No. A-123, “Management's Responsibility for Enterprise Risk 
Management and Internal Control.” Over the next year, the Department will formulate ERM 
plans by coordinating and expanding current risk management activities into an agencywide 
strategy to address its highest priority risks. 


To address capacity for grant, contract, and student financial aid oversight, the Department will 
continue to improve upon its risk-based monitoring planning and provide professional 
development for contract and grant officers. The number and skills of the staff responsible for 
monitoring will be assessed through ERM. 


To improve the usefulness of audits to the Department as well as the audited organization, the 
Department will continue to work toward improving the audit process. The Department is 
developing guidance for grantees on audit readiness, and plans to continue to promote quality 
audits by working with the national auditor community. The Department will continue to improve 
the process for resolving audits and will revise its procedures for audit resolution and closure. 


Objective 6.3: Implementation and Support 
FY 2016 Implementation Strategy 


The OSS within OESE is designed to provide more transparent, higher quality, and better 
differentiated support to states. The matrix organization model adopted by the OSS ensures that 
a state has two primary contacts within the office, and these individuals serve as the liaisons 
across key state-administered grant programs and major federal funding streams that flow to 
each state and district. By consolidating processes and technical assistance, the Department 
will be able to more effectively customize its outreach to individual states and model the critical 
partnerships that states should have with their respective districts. 


In FY 2016, the office focused on continuing professional learning and increasing staff 
knowledge and capacity in the program areas and office functions, as well as supporting states 
in implementing programs administered by the OSS. The OSS is working to deepen staff 
knowledge and build and pilot systems and routines that allow staff to support states with 
implementation through a systemic approach to performance review, policy coordination, data 
review, and technical assistance. In early 2016, the OSS launched staff Professional Learning 
Communities to provide OSS staff opportunities to deepen knowledge of new ESSA provisions. 
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The OSS is also working to create a culture of data use. In FY 2016, OSS staff reviewed and 
validated state data reported on the consolidated state performance report (CSPR). Staff used 
this and other outcome data from the state to better understand state context when conducting 
performance progress review calls. Additionally, leadership used data from the employee 
viewpoint survey, as well as focus group findings, to determine areas of strength and 
weaknesses of the OSS. As a result of these data, leadership began a strategic planning 
process, with input from staff. 


The OSS continued implementing its performance review system. This new system covers all 
OSS programs through a single, streamlined process that encourages SEAs to develop and 
effectively implement integrated and coherent state plans. OSS restructured the comprehensive 
performance review system implementation timeline by elevating support to SEAs preparing for 
full implementation of the ESSA and its requirements in SY 2017-18. As a result, during this 
transition period the OSS implemented, in phases, a comprehensive performance review 
system. 


In FY 2016, OSS implemented several new routines to ensure ongoing coordination with 
internal and external partners. OSS began monthly conversations with the CCSSO to increase 
information sharing and coordination of ESSA transition support to states. Additionally, OSS and 
School Support and Rural Programs have regular meetings to increase coordination and 
communication between state program officers and Department-funded Comprehensive 
Centers. 


FY 2016 Barriers to Success 


Transitioning to the new OSS structure is a significant change that will take time to implement 
fully. OESE and OSS leadership are still establishing new processes and procedures, and the 
transition will take place gradually. Continuing challenges include staffing, appropriate 
professional development and support for staff, and relevant outreach and communication 
internally and externally. Additional challenges for the upcoming year include the launch of an 
updated state performance review and implementing against a new strategic technical 
assistance plan. State capacity to implement new ESSA provisions also continues to be a 
challenge. 


Key Milestones and Future Actions 


In launching the OSS performance review system, the Department deepened its collaborative 
relationship with the states with the quarterly progress checks on a common topic, piloted a risk- 
based fiscal review, conducted several shadowing trips to better understand the work of SEA 
staff, and hosted two collaborative calls that brought several states together to discuss common 
problems and share their approaches. To support continuous improvement of this process, OSS 
surveyed each fiscal review pilot state to gather feedback on the prototype tiered protocol that 
was developed in partnership with the Management and Support Unit of OESE. 


The OSS kicked off the State Support Network, a new four-year, $10 million technical 
assistance contract that will support states as they intervene in the lowest-achieving schools. 
The State Support Network continues to work collaboratively with the Comprehensive Centers 
and other partners to help states. In FY 2017, the State Support Network will provide universal 
support through broadly shared school improvement resources organized in a user-friendly 
website; collective support for technical assistance delivered in person, virtually, and shared by 
multiple organizations; and individual support focused on direct technical assistance from 
providers delivered in person and virtually to address specific state and district needs. 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 97 


PERFORMANCE PLAN SUMMARY 


The OSS restarted an assessment peer review process. After a four-year hiatus, and releasing 
new guidance in fall 2015, the OSS led a peer review process for 38 states in spring and 
summer 2016. From those reviews, the OSS began to develop feedback to states, with the goal 
of providing peer review notes and feedback letters to all reviewed states by the first quarter of 
FY 2017. Having good, actionable data from the assessment system is paramount to having a 
strong accountability system and providing schools, teachers, parents, and the public with the 
information they need to help all kids reach their potential. 


Objective 6.4: Productivity and Performance Management 
FY 2016 Implementation Strategy 


Cybersecurity continues to be a priority at the Department with the implementation of new, and 
the optimization of existing, capabilities to control the flow of sensitive information and prevent 
access to information systems, data, and critical information and infrastructure by unauthorized 
individuals. The ongoing measurement and analysis of cybersecurity incidents and privacy 
breaches, in accordance with OMB and Department of Homeland Security (DHS) guidelines, 
identifies areas for improvement and working with critical stakeholders to implement best 
practices. 


The Department continued its focus on the implementation and utilization of new security tools, 
and fine-tuning existing tools to meet the security needs of the IT environment. Our progress 
demonstrated the effectiveness of the installed capabilities by identifying unauthorized business 
practices and inappropriate handling of sensitive information. The Department increased its 
emphasis on the training of the cybersecurity workforce, to address identified discrepancies, 
and published new standard operating procedures (SOPs). In another effort to expand and 
strengthen its IT security posture, the Department implemented Two-Factor Authentication 
(2FA) for external users of the Grant Management System (G5), in compliance with Homeland 
Security Presidential Directive 12 (HSPD-12). 


The Department continued its implementation efforts to maximize the utilization of electronic 
signature functionality for discretionary and formula grants. The Department performed 
continuous monitoring of discretionary and formula grant activities in the G5 system to evaluate 
adoption of the electronic signature functionality by the program offices. 


The Department focused on applying the lessons learned, testing of the automation changes, 
and implementation of additional process to improve the Department’s overall incident 
response. To reduce the response time, additional resources were assigned and a surge 
capacity has been identified using Department and DHS assets. 


The Department also continues to improve the performance management system to strengthen 
and clarify performance expectations and ensure alignment with organizational goals to support 
a results-oriented performance culture. This effort keeps performance management at the 
forefront of Departmental news on a regular and recurring basis. 


Ensuring staff have the facilities and space to perform is an important part of supporting 
productivity. Through an aggressive strategy of relocating staff and reconfiguring space, as well 
as leveraging wireless connectivity, telework, desk sharing, and “hoteling” seating 
arrangements, the Department achieved the FY 2016 goals in reducing overall footprint (Usable 
Square Feet/USF) and rent costs. The immediate reductions in FY 2016 were mainly due to the 
Rapid Rent Reduction initiative (R3), which compressed personnel into existing Lyndon B. 
Johnson (LBJ) and Potomac Center Plaza (PCP) locations, allowing the release of three 
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commercial leases: 1990 K Street, Capitol Place, and L’Enfant Plaza. This strategy supported 
an overall plan that will further reduce the overall USF and rent costs in FY 2017, FY 2018, and 
beyond. Significant progress was made in FY 2016 that will generate further reductions in space 
and commercial leases. 


FY 2016 Barriers to Success 


During FY 2016, the Department continued to address challenges that included the availability 
of a skilled cybersecurity workforce, and the ability to rapidly implement automated 
cybersecurity capabilities. Additionally, OCIO is working with its IT services provider to provide 
qualified staff and accelerate planned implementations. 


While data assurance and visibility increased, processes and technology continually need to be 
refined to reduce the risk to the Department. The Department met the FY 2016 performance 
target of 234 for IT security incidents and 120 for IT security breaches. To reduce the response 
time, additional resources were assigned and a surge capacity has been identified using 
Department and DHS assets. The Department’s efforts resulted in reducing the cybersecurity 
response time to 22 minutes for the final quarter of FY 2016. 


The use of the electronic signature functionality was heavily promoted during the fourth quarter 
of FY 2016. The outcome was an increase in use by program offices. The Department found 
that only through continued change management efforts will the use of the functionality be fully 
accepted. Change management is an important theme in the Department’s efforts in leadership 
and knowledge management and maintaining a results-oriented performance culture. However, 
employees need time to participate in training opportunities, even online learning opportunities. 
Managers and employees need to be engaged in training and performance management; risks 
are mitigated by senior leadership emphasis and support for the program. Also, risks are 
mitigated by holding managers accountable for completing the process. 


In the areas of space and rent, while the commitment is strong to reduce the amount of space 
and the rent bill, there are several factors that affect the Department’s implementation of these 
plans. One challenge is the availability of funds in the near term, as it will require an initial 
investment to realize the longer-term savings. 


Another challenge is identifying program areas that may grow or shrink in coming years, based 
on both policy and changes in resources and environment. One way to mitigate this is to provide 
flexibilities both in furnishings and layout, but also to leverage increased telework, desk sharing, 
and technology to increase the flexibility and usage of space. 


Key Milestones and Future Actions 


As noted earlier, the Department achieved a 97 percent establishment rate for performance 
plans. The Department continues to make strides toward 100 percent completion of 
performance plans. The involvement and commitment from senior leaders was essential to the 
increase in the percentage of performance plans completed this year and is essential moving 
forward. Performance management and training are critical to employees and supervisors. 
Throughout FY 2016, OM continued to market and educate supervisors on the initial and annual 
requirements for supervisory/managerial training. This effort and the provision of a robust set of 
course offerings for employees will continue in FY 2017. 


In the area of space and rent, the Department will continue with projects and plans to 
consolidate our footprint. The General Services Administration (GSA) is currently performing the 
PCP/LBJ Program of Requirements (POR)/Feasibility study for the renovation, space 
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optimization, and consolidation of a significant portion of the PCP-leased space into the LBJ 
Headquarters Building. The consolidation will reduce the overall utilization rate by reducing 
space allocations in the two locations. Once the Department has the study results, it will 
incorporate the space reductions into its out-year space plan. 
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Management Priorities and Challenges 


The mission of the Department is to promote student achievement and preparation for global 
competitiveness by fostering educational excellence and ensuring equal access. 


Continuous Improvement & Accountability in Department Operations 


To support the work of the Department’s policy agenda, a sound infrastructure and strategic 
allocation of and investments in human resource capacity are critical. The Department uses 
data-driven reviews as a mechanism to bring together the people, resources, and analysis 
needed to drive progress, reinforce priorities, and establish a culture of continuous 
improvement. Also, again in FY 2016, the Department participated in the President’s 
Management Agenda Benchmarking effort and reviewed the data during the Department’s 
FedStat meeting with OMB to discuss organizational performance management, gauge the 
impact of operations, and talk about risk mitigation strategies and the path forward. 


The Department has collaborated with both government and private sector partners to explore 
solutions to payment and data integrity issues, address fraud risk, and provide better oversight 
of its programs. More information on these efforts, including the use of shared services and 
managing enterprise risk, was reported in the FY 2016 Agency Financial Report. 


In FY 2018, the Department’s management priorities will focus on improvements to achieve the 
goals established in the President's Management Agenda. These include: 


1. managing programs and delivering critical services more effectively; 


2. devoting a greater percentage of taxpayer dollars to mission achievement rather than 
costly, unproductive compliance activities; 


3. increasing effectiveness and efficiency in supporting program outcomes; and 


4. increasing accountability for improving performance. 


Addressing Management Challenges 


As summarized in the FY 2016 Agency Financial Report, the Office of Inspector General (OIG) 
has identified five management challenges for the Department in FY 2017. These challenges 
are detailed in the FY 2017 Management Challenges Report. 


In FY 2016, the Department began a new effort to address these challenges. The Office of the 
Deputy Secretary initiated a detailed review of the five management challenges, assigned 
senior managers to be accountable for each, and assembled a workgroup of other senior 
managers throughout the Department to address the noted challenges. This effort underway 
has helped identify systemic root causes to ensure that the Department’s actions produce 
results. The OIG has stated that it considers this initiative to be a positive step towards 
addressing long-standing management challenges and encouraged the Department to continue 
to explore approaches that result in targeted focus within each of the areas. 


While these challenges reflect continuing vulnerabilities and emerging issues, the Department 
remains committed to improved governance and better business processes. Management has 
worked closely with OIG to gain its perspective about the Department’s most significant 
management and performance challenges. 
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In FY 2018, the Department will continue implementing coordinated actions to address the root 
causes of management challenges. The Government Performance and Results Modernization 
Act of 2010 requires the Department to include in its APP the following information on those 
planned actions, including performance goals, indicators, and milestones, to address these 


challenges: 


Management 


Challenge 


PAVererolU] ay t-le)(s) 
Official 


na Fclalarcye mm Ave1i (0) ats) 


Performance Goals/Indicators/ 


Milestones 


workforce 

Develop ED Cybersecurity 
Workforce Development Plan 
to address gaps identified in 
the ED cybersecurity 
workforce assessment 
Establish Department/FSA 
Cybersecurity Committee 
Develop ED Cybersecurity 
Strategy and Implementation 
Plan (ED-CSIP) 


Improper Chief " Continue to coordinate with " Goal: Minimize the risk of 
Payments Financial OMB on refinements to improper payments without 
Officer (FSA) estimation methodology for unduly burdening students 
student aid programs « Indicators: Strengthened 
" Continue to improve grant and controls, improved risk 
contract risk assessments assessments and estimates 
* Continuous monitoring of = Key milestones: Engage 
processes and systems to OMB and Congress, revise 
strengthen controls, including risk assessment and 
work with IRS to implement estimation methodologies, 
system upgrades and implement actions intended 
reactivate the Data Retrieval to help achieve FY 2018 
Tool to enable applicants to targets including reactivation 
securely obtain necessary of IRS Data Retrieval Tool 
financial information to 
complete the FAFSA or apply 
for an income-driven 
repayment (IDR) plan 
Information Chief = Complete ED cybersecurity " Goal: Mature the 
Technology Information workforce assessment to Department’s cybersecurity 
Security Security identify current gaps in operations to achieve the 
Officer (OCIO) Department cybersecurity next level of effectiveness as 


defined by the FISMA 
maturity model 

Indicators: Updated plans 
and strategies, tools 
acquired, increased detection 
and reporting of system 
breaches 

Key milestones: FY 2018 and 
FY 2019 budget submissions, 
deployment phases through 
FY 2018 


Oversight and 
Monitoring 


Director, Risk 
Management 
(FSA) 


Increase expertise among 
Department staff to effectively 
monitor student aid program 
participants and grant 
recipients 

Improve monitoring and 
support processes, including 
increased information-sharing 
across offices and more 
integrated and targeted 
monitoring approaches 
Expand data analysis 
capabilities and develop better 


Goal: Improve program 
compliance and grant 
outcomes through high- 
quality and timely monitoring 
and support 

Indicators: Increased use of 
data to identify higher-risk 
program participants, 
improved internal controls to 
ensure efficient and effective 
monitoring, improved 
customer feedback 
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Performance Goals/Indicators/ 


Milestones 


ways to identify higher-risk 
program participants to detect 
potential misuse of funds 
Highlight areas of ambiguity 
or common misunderstanding 
for program participants and 
provide additional technical 
assistance in those areas 


Key milestones: update risk 
assessments for student aid 
participants; gather feedback 
from annual student aid 
conference attendees; 
implement FY 2018 
Enterprise Risk Management 
requirements; implement risk- 
based, tiered, and virtual 
monitoring approaches for 
grant recipients; strengthen 
electronic records for grant 
monitoring; develop skills in 
staff 


Data Quality 
and Reporting 


Director, Risk 
Management 
Service (ODS) 


Implement tracking tool using 
key elements from 
Consolidated State 
Performance Reports (CSPR) 
to improve monitoring and 
correction of data quality 
issues 

Finalize data management 
strategy and continuation with 
principal offices stewarding 
data quality, as part of the 
EDFacts Data Governance 
Board 

Adopt new standard 
certification language in each 
data collection system; require 
states to provide additional 
justification for questionable 
data 

Identify promising practices in 
monitoring grantee controls 
over data accuracy and share 
across principal offices 
Leverage Single Audits to 
help assess grantee data 
quality 


Goal: Strengthen data quality 
and reporting to better enable 
evidence-based decision- 
making 

Indicators: Improved 
governance of data quality, 
fewer data quality audit 
findings, increased use of 
data to improve program 
outcomes 

Key milestones: Expand use 
of data tracking tool, 
disseminate promising 
practices, revise FY 2018 
Compliance Supplement 


Information 
Technology 
System 
Development 
and 
Implementation 


Director, 
Information 
Technology 
Program 
Services 
(OCIO) 


Establish a governing OCIO- 
based Project Management 
Office capability to begin 
integrating and optimizing 
resources to review project 
work plans, methods, and 
deliverables and provide 
support, review, and oversight 
to Department IT project 
managers 

Consolidate and standardize 
current LCM/LMM structure 
and governance to monitor, 


Goal: Mature the 
Department’s compliance 
with FITARA requirements 
Indicators: Improved 
governance and capabilities, 
fewer audit findings 

Key milestones: Implement 
new governance structure 
and LCM/LMM reporting, 
target dates through FY 2018 
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Management | Accountable Performance Goals/Indicators/ 


Challenge Official ponmee a CuCns Milestones 


manage, and enforce IT 
project management practices 
and IT solution delivery 
methods 

= Educate systems owners on 
LCM/LMM processes by 
forming an internal community 
of practice 

" Develop Department 
Guidance on Software 
Development Best Practices 


Lower-Priority Program Activities 


The Cuts, Consolidations and Savings volume of the President’s Budget identifies the lower- 
priority program activities, where applicable, as required under the GPRA Modernization Act of 
2010, 31 U.S.C. 1115(b)(10). The public can access the volume at 


http://www.whitehouse.gov/omb/budget. 
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APPENDICES 


Appendix A: Data Validity and Verification 


The Government Performance and Results Act Modernization Act of 2010 requires 
agencies to describe the accuracy and reliability of data presented. The Department 
continues to strengthen its approach to data verification and validation. Details of 
how the Department assesses the completeness and reliability of the data reported 
are presented as part of this appendix, and known limitations of the data are also 
included. 


To the best of my knowledge, the data verification and validation process and the 
data sources used provide, to the extent possible, complete and reliable 
performance data pertaining to goals and objectives in our FY 2014—18 Strategic 
Plan. Through a process of continuous improvement, the Department regularly 


assesses its validation process and welcomes input from stakeholders. 


ee 


Joseph C. Conaty 
Delegated the Duties and Functions 
of the Deputy Secretary 
May 23, 2017 
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GOAL 1: POSTSECONDARY EDUCATION, CAREER AND TECHNICAL EDUCATION, AND ADULT EDUCATION. Increase college access, affordability, quality, 


F- Tato mere)sa)e)(=it(e]aM onvmllanle)con//are Mm ofel-)t1-Yere) alel-lavm-\e [0 (er-1i(o)a m= ale M lic=\elare Mm (=r-laallare me) e) eLe)acelalit(-smm ce )mnveleliatcw-lalem- (00) [tom 


Metric (Indicator) Data Quality, Limitations and Improvements 


Federal student loan delinquency rate Federal Student Aid (FSA) Data Center NSLDS is currently the source of the data. Verification and validation occur in three 
aspects of the process: 

(1) Data providers (servicers in this case) transmit data to NSLDS where the data are 
subject to edits and screening. FSA works with the servicers to resolve anomalies. 

(2) Users who analyze the data sometimes identify anomalies in the course of their 
analytic roles and work with experts in NSLDS to resolve. In some cases NSLDS will 
again work with the servicer(s) to resolve. 

(3) The report providing the data for this metric has been validated. This means that 
the programming and logic have been independently reviewed. 


Web traffic to the College Scorecard Google Analytics data from College The data are simply pulled from the GSA Google Analytics account. The data are 
Scorecard checked for reasonability, and incorporated alongside the existing baseline data. The data 
need to be matched with baseline data and collated accordingly. There is limited analysis 
required. 


The data will be matched with prior data for reasonability checks. The Office of Planning, 
Evaluation and Policy Development will pursue solutions to anomalous data, but do not 
anticipate a significant problem, as the data are produced relatively automatically. 


Percent of High School Seniors Filing a || The denominator is the number of The denominator is the number of graduating seniors according to the most recent 
FAFSA graduating seniors according to the most projection by National Center for Education Statistics (3,365,560). 
recent projection by NCES (Table 219.10 Number of seniors obtained from the most recent release of Digest of Education Statistics 
within the Digest of Education Statistics). (2013) obtained here: http://nces.ed.gov/programs/digest/d13/tables/dt13_219.10.asp 
The numerator is from Federal Student Aid’s 
(FSA’s) Central Processing System and is The numerator is based on the number of applications during the first nine months of the 
based on the number of applications during application cycle that are—as of September 30 of the first year of the application cycle— 
the first nine months of the application cycle complete (not rejected; first-time filers; incoming freshmen, with or without previous 
that are—as of September 30 of the first college attendance; age 18 or less as of June 30 of the first year of the application cycle; 
year of the application cycle—complete (not reporting high school diploma attainment; and attended a high school in the 50 states and 
rejected); first-time filers; incoming DC. (Data Source: FSA’s Central Processing System. Query timeframe: January— 
freshmen, with or without previous college September 30, 2014: 2,021,691) 
attendance; age 18 or less as of June 30 of 
the first year of the application cycle; Broadly speaking, several individuals and organizations serve as controls, including the 
reporting high school diploma attainment; Customer Analytics Group (for primary calculation), Business Intelligence (for technical 
and attended a high school in the fifty states || validation), and the Financial Reporting and Analysis Branch for ensuring documentation 
and Washington, DC. is complete and archived. 


In addition to the above controls, queries and calculations are simultaneously conducted 
on data from previous years to ensure technical definitions remain consistent and 
calculations are accurate. 


Being that the Central Processing System is the core legacy system used to determine 
aid eligibility and awards and understanding the significance, value and reliability of the 
Digest of Educational Statistics, FSA feels confident that the outcomes are reliable, 
accurate, and valid. 


*Metric associated with an Agency Priority Goal. 
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| Metric No. | Metric (Indicator) Data Quality, Limitations and Improvements 


— 
a 
a 
a 
P| 
a 


Index of national annual aggregate 
earnings of VR consumers (based on 
the number of competitive integrated 
employment outcomes, hours worked, 
and hourly wages of VR Consumers) 


Index of national annual aggregate 
earnings of Transition-Age Youth 
(based on the number of competitive 
integrated employment outcomes, 
hours worked, and hourly wages of VR 
Transition-Age Youth) 

*Number of data points or other 


information reports released on the 
FSA Data Center 


Pell enrollment at IHEs with high 
graduation rates 


Number of states that develop or 
strengthen career pathways policies, 
guidance, and or legislation 


*Degree attainment among 25-34-year- 
old age cohort 


*Metric associated with an Agency Priority Goal. 


Rehabilitation Services Administration 
(RSA)- 911 


FSA Data Center 


Data from College Scorecard 


Development data from the National Skills 
Coalition, possibly supplemental with data 
from the Association of State Legislators and 
CLASP 


NCES Digest of Education Statistics, Table 
104.30 
(https://nces.ed.gov/programs/digest/d15/tab 
les/dt15 104.30.asp), Number of persons 
age 18 and over, by highest level of 
educational attainment, sex, race/ethnicity, 
and age: 2015. Tabulated from Current 
Population Survey data, U.S. Census 
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Each grantee’s RSA-911 is submitted as a text file to RSA via an SFTP server. We run an 
edit program that converts the text file into MS Access and verifies the accuracy of the 
data. Each of the 215 data elements is verified. If errors are identified by the application, 
the grantee is requested to correct the errors and resubmit the data. Once the data file is 
corrected and resubmitted, it is checked again using the same process. If anomalies are 
identified, the grantee must provide an explanation and verify that the data is correct. 


RSA provides the agencies with the edit program that we use and are encouraged to run 
their data using the program often, at least quarterly, during the year. In addition, 
agencies have edits in their own systems which run additional checks. 


The Validation Subject Matter Expert (SME) works with the Request SME to understand 
the solution used to fulfill the request, checking all results, code, and documentation 
produced by the Request SME. Additionally, the Validation SME ensures there are no 
data disclosure issues that need to be addressed. In the event the Validation SME cannot 
validate the request results, they contact the Request SME and the Data Request Team 
(DRT) to resolve any issues necessary to successfully complete their validation. The 
Validation SME completes their portion of the Request Template and notifies the DRT 
that the validation step is complete. Upon receipt of the data that was requested, the DRT 
confirms with the assigned Validation SME that the results have been validated and 
documented per their selected solution and enters it into the Data Request Tracking 
database. 


Validation of the data is conducted by the National Center for Education Statistics (NCES) 
through its annual IPEDS data validation process. NCES will ensure accuracy, while 
Office of Planning, Evaluation and Policy Development / Policy and Program Studies 
Service will incorporate the data into the Scorecard datasets. 


Analysis will be conducted to identify anomalous data. They will be handled on a case-by- 
case basis, as most anomalies are due to variation from year-to-year or small n-size and 
are already pooled and suppressed before publishing the data. Other data problems may 
necessitate additional runs, qualifications, or suppression for stability reasons. 


The Department cannot independently verify the data, but rely on the tracking of the 
National Skills Coalition. There are no processes in place to check for anomalous data. 


Data quality and limitations are documented in: 
http://www.census.gov/prod/2006pubs/tp-66.pdf. 


NCES tabulates the data, which is verified prior to publication according to NCES 
guidelines. 
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Metric (Indicator) Data Quality, Limitations and Improvements 
Enrollment in IHEs below the minimum _ || Data from College Scorecard Tool Extensive validation is conducted by the Treasury Department before it is shared with the 
earnings threshold Department of Education, both on the earnings file used to construct the measure and on 
the resulting data file by institution. The Department will also continue to validate the unit- 
level files that are shared with the Treasury Department to generate the match. 


Analysis will be conducted to identify anomalous data. They will be handled on a case-by- 
case basis, as most anomalies are due to variation from year-to-year or small n-size and 
are already pooled and suppressed before sharing with the Department of Education. 
Other data problems may necessitate additional runs, qualifications, or suppression for 
stability reasons. 


The median earnings is calculated only on individuals who are employed (i.e., have 
earnings greater than zero). The data are not produced for small years, and are pooled 
across years, to reduce variability or misrepresentative figures. 
Persistence among first-time filing aid FSA’s Common Origination and Data used for these calculations are based on counts from operational systems (number 
recipients Disbursement (COD) system of recipients and number of applicants). Moreover, standardized queries are used to re- 
run and match calculations for earlier cycles. Therefore, the metric as defined should be 
considered verified and valid. 


Data used for these calculations are based on counts from operational systems (number 
of recipients and number of applicants meeting certain criteria). Anomalous data would 
suggest there are recipients that are not recipients (or vice versa) or applicants that are 
not applicants (or vice versa). A variety of internal controls are in place tracking both of 
these processes. 


*Metric associated with an Agency Priority Goal. 
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Metric (Indicator) Data Quality, Limitations and Improvements 


Number of STEM postsecondary Integrated Postsecondary Education Data IPEDS collects completions by Classification of Instructional Programs (CIP) Code which 
credentials awarded System (IPEDS) Data Center may be aggregated into STEM and non-STEM counts. The Department of Education 
does not currently have a single definition for which CIP codes are STEM. 


Doctor’s Degrees: (324.25, 324.30 and 

324.35) Data quality and limitations are identified in IPEDS First Look Publications, “Data 
http://nces.ed.gov/programs/digest/d14/table |} Collection Procedures,” and IPEDS methodology available at 

s/dt14 324.25.asp http://nces.ed.gov/pubs2014/2014067.pdf. Data are checked by NCES for consistency. 
http://nces.ed.gov/programs/digest/d14/table 

s/dti4 324.30.asp 

http://nces.ed.gov/programs/digest/d14/table 

s/dt14 324.35.asp 


Master’s Degrees: (323.30, 323.40, 323.50) 
http://nces.ed.gov/programs/digest/d14/table 
s/dti4 323.30.asp 
http://nces.ed.gov/programs/digest/d14/table 
s/dt14 323.40.asp 
http://nces.ed.gov/programs/digest/d14/table 


s/dt14_323.50.asp 


Bachelor’s Degrees: (322.30, 322.40, 
322.50) 
http://nces.ed.gov/programs/digest/d14/table 
s/dt14 322.30.asp 
http://nces.ed.gov/programs/digest/d14/table 
s/dt14 322.40.asp 
http://nces.ed.gov/programs/digest/d14/table 
s/dt14_ 322.50.asp 


Associate’s Degrees: (321.30, 321.40, 
321.50) 
http://nces.ed.gov/programs/digest/d14/table 
s/dt14 321.30.asp 
http://nces.ed.gov/programs/digest/d14/table 
s/dt14 321.40.asp 
http://nces.ed.gov/programs/digest/d14/table 


s/dti4_321.50.asp 


Certificates (320.20) 
http://nces.ed.gov/programs/digest/d14/table 


s/dt14 320.20.asp 


*Metric associated with an Agency Priority Goal. 
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GOAL 2: ELEMENTARY AND SECONDARY EDUCATION. Improve the elementary and secondary education system’s ability to consistently deliver excellent 
Takei dale: (o)ale-lite]al-xe MUI (amare ce) cele t-m-er-(e(-Vaal (omc t-laver-lcet-MUVali(-m elxenUcellaleM=1ic-xe1t\(-Mc1e] o) ee) am@cr-) aj (er-icm (oMes(ol-t-M-(eral(-\V/-\na(-lalm@r-lace me) e) ele) alaliavmet-| oy-Mmr-laleM-Vali0Li-M-|/Boitee(- 1g] 6 


fo] ¢-YolUr-1X- walle] amcxeavele) Mexe)|{-ve{- wear lao Mer-|¢-v-1 eng -t- (0) 


Metric (Indicator) Data Quality, Limitations and Improvements 


Sinadclge sere) of states / territories that have 
STS | eine endeweereaty | college- and career-ready 
standards 

Number of states / territories that are 
implementing next-generation reading 
and mathematics assessments, aligned 
with college- and career-ready standards 


(ESEA) Monitoring 


Disparity in the rates of out-of-school 
suspensions for students with disabilities 
and youth of color (youth of color metric) 


Civil Rights Data Collection (CRDC) 


Disparity in the rates of out-of-school 
suspensions for students with disabilities 
and youth of color (SWD, IDEA only 
metric) 


Number of states that have fully ESEA Flexibility Applications and 
implemented teacher and principal Monitoring 

evaluation and support systems that 

consider multiple measures of 

effectiveness, with student growth as a 

significant factor 
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yamorrey and Secondary Education Act 


The Office of Elementary and Secondary Education’s (OESE) Office of State Support will 
count the number of states that either (a) have a memorandum of understanding in place 
to implement standards common to a number of states or (b) have a letter from 
Institutions for Higher Education in their state certifying that their state has college- and 
career-ready standards. There are no known data limitations. 


The Office of Elementary and Secondary Education’s (OESE) Office of State Support 
(OSS) monitors that states are meeting requirements under the ESEA waiver. Monitoring 
includes desk monitoring and on-site monitoring. Previously, OSS state leads monitored 
using a monitor protocol and rubric based on ESEA Flexibility waivers to ensure that 
monitoring is consistent across all states with ESEA Flexibility. OSS is revising its 
monitoring practices based on passage of the ESEA, as amended by the ESSA, in 
December 2016, based on statutory revisions, as well as regulatory action. OSS state 
leads work with their Group Leader to finalize monitoring reports. There are no known 
data limitations. 


The Department is no longer conducting ESEA Flexibility monitoring, but states continued 
to implement their ESEA Flexibility requests through August 1, 2016, before beginning the 
transition to the ESSA. 


In December 2015, the President signed into law the Every Student Succeeds Act 
(ESSA), which reauthorizes the Elementary and Secondary Education Act of 1965 
(ESEA). Under the ESEA as amended by the ESSA, the Secretary of Education is 
prohibited from, among other prohibitions, prescribing “any aspect or parameter of a 
teacher, principal, or other school leader evaluation system within a State or local 
educational agency” and “indicators or specific measures of teacher, principal, or other 
school leader effectiveness or quality.” The Department has no new data on the number 
of states implementing teacher and leader evaluation and support systems consistent 
with this performance metric and no way to verify the data reported. 


The Department is no longer conducting ESEA Flexibility monitoring, but states continued 
to implement their ESEA Flexibility requests through August 1, 2016, before beginning the 
transition to the ESSA. 


The Office for Civil Rights (OCR) strives to ensure CRDC data are an accurate and 
comprehensive depiction of student access to educational opportunities in school 
districts. The submission system includes a series of embedded edit checks to ensure 
significant data errors are corrected before the district submits its data. Additionally, each 
district is required to certify the accuracy of its submission. Only a district superintendent, 
or the superintendent’s designee, may certify the CRDC submission. Ultimately, the 
quality of the CRDC data depends on accurate collection and reporting by the 
participating districts. 
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*Number of persistently low graduation NCES EDFacts EDFacts works with the data stewards to determine the appropriate business rule checks for 
rate high schools these data. The checks that were done on these data include: 

. File validation and format checks: Identifies file submissions that do not conform 
to the accepted file format, the data universe (school/LEA/state directory), or the 
reporting requirements (mandatory reporting fields). 

Submission Edit Business Rules: Submission edits and warnings ensure that the 
data meet or exceed an acceptable level of reasonability by checking the values 
entered in a field against other similar values in the same file or across files. If a 
discrepancy is found (i.e., a value falls outside of the acceptable range), a 
submission error or warning is issued. Unlike format and validation edits, 
submission edits and warnings are programmed into the EDFacts Submission 
System and applied to the data after they are in the staging database. 
Coordinated Data Quality Reviews (CDQR): Identifies potential errors and 
anomalies related to completeness, consistency, and comparability in the file 
submissions that would affect the quality and usability of data in the files. 

o | CDQR Process: At the end of a collection period, EDFacts runs checks to 
validate data quality of submitted data and presents all results to program 
offices. It is the responsibility of the program offices, in consultation with 
the EDFacts staff, to determine which identified errors to escalate to 
states for further review. The EDFacts Partner Support Center (PSC) 
sends these errors to states for remediation, which take the form of 
explanations for data anomalies, data file submission updates, and data 
corrections. OESE should be considered the point-of-contact for 
identifying which errors were escalated and the result of those 
escalations. 


Data concerns for state, district and school level 2014—15 data submissions are documented 
and available in two places: 
State Notes to submitted Adjusted Cohort Graduation Rate (ACGR) data are 
available within the Department's ED Data Express Tool at 
http://eddataexpress.ed.gov 
Identified Data Anomalies are documented at the state level in appendix B to the file 
documentation released along with school and district level data files at 


http://www2.ed.gov/about/inits/ed/edfacts/data-files/index.html 


Additional concerns, if any are identified, related to the calculated national rate or work done 
prior to 2014—15 will be available in future releases of the NCES report “Public High School 
Four-Year On-Time Graduation Rates and Event Dropout Rates: School Year ####-##.” The 
most recent report covers 2010-11 and 2011-12. Reports covering more recent years of data 
are being prepared and once released will be available at 
http://nces.ed.gov/ccd/pub_dropouts.asp. 


Path to public release and national rates: For each collection cycle states report their calculated 
cohort counts and ACGR graduation rates. These counts and rates are tested against a number 
of submission edit business rules and the ACGR Coordinated Data Quality Review for format, 
consistency, completeness, and comparability. Those rule checks are delivered to the Office of 
Elementary and Secondary Education (OESE) for follow-up correspondence through the 
EDFacts Partner Support Center with the states. The program office, in consultation with NCES 
EDFacts staff and other stakeholders, meet to determine which issues identified by the 
business rules should be raised with the state for explanation, update, or correction. Following 
that review, data are then aggregated to the national level for the purpose of calculating and 
publishing a national rate. NCES processes these data to force conformity of reporting 
categories (mapping reported “Major Racial/Ethnic Groups” to the traditional 5 racial/ethnic 
groups) and addresses any issues raised by missing data. After imputations are made, NCES 
produces a national rate for the country as a whole; a rate which is representative of every 
state. Once produced, NCES documents any remaining issues with these data, and the 
aggregation and imputation methodology in a public report. That report, documentation, and the 
associated data tables are put through several stages of review including independent reviews 
*Metric associated with an Agency Priority Goal. at the division (Administrative Data), center (NCES), and principal office (IES) levels prior to 
public dissemination. 
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WM Ceca tom (Lae L(er-1 Col g) Data Source Data Quality, Limitations and Improvements 


Percentage of SIG schools in Cohort 5 Analytic dataset produced by the To produce these numbers, OESE relies on the analytic dataset produced by the 
that are above the 25th percentile in contractor for the SIG National Summary, contractor for the SIG National Summary, because this provides a list of SIG schools and 
mathematics, as measured by their state || because this provides an accurate list of flags for different exclusions that we include in the analysis. The analytic dataset is a 
assessments SIG schools and flags for different combination of EDFacts student achievement files in Math and Reading, the NCES 
Percentage of SIG schools in Cohort 5 exclusions that we are included in the Common Core of Data, SIG lists provided to EDFacts by OSS, and Exclusions that are 
that are above the 25th percentile in analysis. (The analytic dataset is a generated by the contractor that apply to these results. 
reading language arts, as measured by || Combination of EDFacts student noes 
their state assessments achievement files in Math and Reading, For the 2012-13 data, a draft version of the analytic dataset was available in March of 
the NCES Common Core of Data, SIG lists |} 2015, although OESE hopes that it will be available sooner in future years. 
provided to EDFacts by Office of State 
Support (OSS), and Exclusions that are Information can be found on the EDFacts website 
generated by the contractor that apply to (http://www2.ed.gov/about/inits/ed/edfacts/index.html). 
these results.) 


Percentage of high school and middle Schools and Staffing Survey (SASS), The methods report for the 2011-12 SASS is not yet released. Study documentation from 
school teachers who teach STEM as NCES the 2007-08 survey is available at https://nces.ed.gov/surveys/sass/methods.asp. 

their main assignment who hold a 

corresponding undergraduate degree 


Number of External high school College Board/AP administrative records College Board Public School List is updated annually by state DOEs; thus small changes 

graduates who have taken at least one to the list over time are to be expected as schools open, close, and/or merge. Students 

STEM AP exam are assigned to graduating cohorts based on self-reported information (i.e., grade level 
and/or graduation year) provided at the time of registration (in the case of SAT) or test 
administration (in the case of AP and PSAT). The College Board matches students’ data 
across programs to identify the most recent valid value when assigning students to 
cohorts. 


*Metric associated with an Agency Priority Goal. 
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GOAL 3: EARLY LEARNING. Improve the health, social-emotional, and cognitive outcomes for all children from birth through 3rd grade, so that all children, 


rex=]at(ore)t-la\ am taress1-M Zia male] am al=\-10t-Mmr-1K-Meolam ie: le. @colmele-(olUr-ldlalemicelaamalie|amcve:aceye) Mere)|(-1e(-tar- lave mer-l¢-1-1ear-1-\e ha 


Metric (Indicator) Data Source Data Quality, Limitations and Improvements 


3.1.A *Percent of 4-year olds enrolled in state Preschool Development Grants Annual 
preschool programs Performance Reports (APR), Data 
Workbook, Table A(1)(a) and (b) 


3.1.B *Number of states with high-quality National Institute for Early Education 

preschool programs standards Research (NIEER) Yearbook (The State of 
Preschool), Table 1: State Ranking and 
Quality Checklist Sums, Column “ Quality 
Checklist Sums” 

3.2.A Number of states that require a teacher National Institute for Early Education 
with a bachelor’s degree in a state Research (NIEER) Yearbook (The State of 
preschool program Preschool), Workforce Supplement, Table 

3: Preschool Teacher Qualifications and 
Salary, Column “BA Required” 


*Metric associated with an Agency Priority Goal. 
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APR data is collected externally. The data are collected primarily through a performance 
report template and a data workbook submitted by State Project Directors for the grants. 


Data are collected by program offices through a technical assistance contractor using a 
performance report template and self-administered electronic submission tool. The data 
collection period for one year is reported in the following year’s APR submission. For 
example, school year 2015-2016 data is reported in February of 2016. During November 
of 2015, State Project Directors are provided with any submission updates and granted 
access to the APR submission tool. The APR data collection period is January 1, 2015— 
December 31, 2015. The Technical Assistance (TA) contractor performs an analysis and 
data anomalies are clarified and verified with the State Project Director. 


NIEER data are collected externally. The data are collected primarily through surveys of 
state preschool administrators using the State of Preschool data collection instrument. 
Data are collected by staff at Rutgers University’s NIEER using a self- administered 
online data instrument. The collection for the 2013-14 school year, for example, began in 
the fall of 2013. During November of 2013, links to the web-based survey instrument were 
sent to administrators of the state-funded preschool programs. The initial listing of 
administrators was drawn from the 2011-12 State of Preschool data collection. NIEER 
project staff updated the list, where appropriate, by recontacting state agencies to 
determine if new programs had been started since the 2013-14 school year, or whether 
any programs had been left out of the previous report (no such programs were identified). 
All states and the District of Columbia responded. Data collection ended in April 2015. All 
programs included in the data collection and current report are those that are funded and 
directed by the states to support group learning experiences for preschool-age children, 
usually ages 3 and 4. A full list of criteria programs must meet for inclusion is available in 
the introduction of this report. Please see Data Verification Overview. 
http://nieer.org/yearbook. 


NIEER data are collected externally. The data are collected primarily through surveys of 
state preschool administrators using the State of Preschool data collection instrument. 
Data are collected by staff at Rutgers University’s NIEER using a self- administered 
online data instrument. The collection for the 2013-14 school year, for example, began in 
the fall of 2013. During November of 2013, links to the web-based survey instrument were 
sent to administrators of the state-funded preschool programs. The initial listing of 
administrators was drawn from the 2011-12 State of Preschool data collection. NIEER 
project staff updated the list, where appropriate, by recontacting state agencies to 
determine if new programs had been started since the 2013-14 school year, or whether 
any programs had been left out of the previous report (no such programs were identified). 
All states and the District of Columbia responded. Data collection ended in April, 2015. All 
programs included in the data collection and current report are those that are funded and 
directed by the states to support group learning experiences for preschool-age children, 
usually ages 3 and 4. A full list of criteria programs must meet for inclusion is available in 
the introduction of this report. Please see Data Verification Overview. 
http://nieer.org/yearbook. 
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Werarem are lersttera) | Data Source Data Quality, Limitations and Improvements 


Number of states collecting and Race to the Top (RTT)-Early Learning The data are limited to the 20 Race to the Top-Early Learning Challenge (RTT-ELC) 
reporting disaggregated data on the Challenge (ELC) Technical Assistance states. 

status of children at kindergarten entry Center 
using a common measure 


Department staff has requested Kindergarten Entry Assessment (KEA) data be collected 
in the 2015-16 academic year as part of the State of Preschool data collection and has 
requested to add KEA data reporting in EDFacts. In addition, the new Preschool 
Development Grants will provide data on additional states. 


*Metric associated with an Agency Priority Goal. 
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: EQUITY. Increase educational opportunities for underserved students and reduce discrimination so that all students are well-positioned to succeed. 


| Metric No. | Metric (Indicator) Data Source Data Quality, Limitations and Improvements 


*National high school graduation rate NCES EDFacts 


*Metric associated with an Agency Priority Goal. 
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EDFacts works with the data stewards to determine the appropriate business rule checks for 
these data. The checks that were done on these data include: 

. File validation and format checks: Identifies file submissions that do not conform 
to the accepted file format, the data universe (school/LEA/state directory), or the 
reporting requirements (mandatory reporting fields). 

Submission Edit Business Rules: Submission edits and warnings ensure that the 
data meet or exceed an acceptable level of reasonability by checking the values 
entered in a field against other similar values in the same file or across files. If a 
discrepancy is found (i.e., a value falls outside of the acceptable range), a 
submission error or warning is issued. Unlike format and validation edits, 
submission edits and warnings are programmed into the EDFacts Submission 
System and applied to the data after they are in the staging database. 
Coordinated Data Quality Reviews (CDQR): Identifies potential errors and 
anomalies related to completeness, consistency, and comparability in the file 
submissions that would affect the quality and usability of data in the files. 

o  CDQR Process: At the end of a collection period, EDFacts runs checks to 
validate data quality of submitted data and presents all results to program 
offices. It is the responsibility of the program offices, in consultation with 
the EDFacts staff, to determine which identified errors to escalate to 
states for further review. The EDFacts Partner Support Center (PSC) 
sends these errors to states for remediation, which take the form of 
explanations for data anomalies, data file submission updates, and data 
corrections. OESE should be considered the point-of-contact for 
identifying which errors were escalated and the result of those 
escalations. 


Data concerns for state, district and school level 2014—15 data submissions are documented 
and available in two places: 
State Notes to submitted Adjusted Cohort Graduation Rate (ACGR) data are 
available within the Department’s ED Data Express Tool at 
http://eddataexpress.ed.gov 
Identified Data Anomalies are documented at the state level in appendix B to the file 
documentation released along with school and district level data files at 
http://www2.ed.gov/about/inits/ed/edfacts/data-files/index.html 


Additional concerns, if any are identified, related to the calculated national rate or work done 
prior to 2014—15 will be available in future releases of the NCES report “Public High School 
Four-Year On-Time Graduation Rates and Event Dropout Rates: School Year ####-##.” The 
most recent report covers 2010-11 and 2011-12. Reports covering more recent years of data 
are being prepared and once released will be available at 


http://nces.ed.gov/ccd/pub_dropouts.asp. 


Path to public release and national rates: For each collection cycle states report their calculated 
cohort counts and ACGR graduation rates. These counts and rates are tested against a number 
of submission edit business rules and the ACGR Coordinated Data Quality Review for format, 
consistency, completeness, and comparability. Those rule checks are delivered to the Office of 
Elementary and Secondary Education (OESE) for follow-up correspondence through the 
EDFacts Partner Support Center with the states. The program office, in consultation with NCES 
EDFacts staff and other stakeholders, meet to determine which issues identified by the business 
rules should be raised with the state for explanation, update, or correction. Following that review, 
data are then aggregated to the national level for the purpose of calculating and publishing a 
national rate. NCES processes these data to force conformity of reporting categories (mapping 
reported “Major Racial/Ethnic Groups” to the traditional five racial/ethnic groups) and addresses 
any issues raised by missing data. 
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| —sMetric (Indicator) = (Tate [eto 9) ck | Data Source Data Quality, Limitations and Improvements 


*Gap in the graduation rate between all 
students and students from low-income 
families 


*Number of schools that do not have a 
gap or that decreased the gap between 
students from low-income families and 
the state average for all students 


Percentage of proactive civil rights Office for Civil Rights’ (OCR) Case 
investigations launched annually that Management System (CMS) and 
address areas of concentration in civil Document Management (DM) systems 
rights enforcement 


Percentage of proactive civil rights 
investigations resolved annually that 
address areas of concentration in civil 
rights enforcement 


*Metric associated with an Agency Priority Goal. 
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After imputations are made, NCES produces a national rate for the country as a whole; a rate 
which is representative of every state. Once produced, NCES documents any remaining issues 
with these data, and the aggregation and imputation methodology in a public report. That 
report, documentation, and the associated data tables are put through several stages of review 
including independent reviews at the division (Administrative Data), center (NCES), and 
principal office (IES) levels prior to public dissemination. 


The verification and validation of CMS data utilize (1) rules built into the CMS Case 
Management component; (2) periodically checks on questionable data; (3) guidance and 
reference on data entries; and (4) regular training and guidance for primary users 
imputing the data. 


(1) Rules built in CMS: 
Samples of CMS validations: 
- Docket number is assigned by the CMS based on four required data (Active 
Office, Case Type, Institution Type, and Case Open Date). 
- Date should be a validated date. 
- When a notification letter is issued, current stage is changed from Evaluation to 
Investigation. 
- Resolution Type is filled by the CMS based on the Resolution Codes of allegations. 


(2) Periodic checks on questionable data: (monthly) 
- Missing entries: such as cases resolved without (a) case resolved/closed dates, (b) 
allegations and (c) resolutions. 
- Inconsistent entries: such as resolution type not matched with resolution code. 


(3) Guidance and reference materials for use by primary users making data entries are 
posted at the OCR SharePoint site and are available to all OCR staff: 
- Lists of issue codes, specific bases, resolution codes, milestones, stage, data 
definition, and etc. 
- CMS Data Entries requirements and checklist. 
- CMS User Manual. 


(4) Users Training: 
- User training is available as needed. OCR also provides training through a “peer 
helper” or “train the trainer” model for staff in the field offices and headquarters. 
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GOAL 5: CONTINUOUS IMPROVEMENT OF THE U.S. EDUCATION SYSTEM. Enhance the education system’s ability to continuously improve through better 


and more widespread use of data, research and evaluation, evidence, transparency, innovation, and technology. 


| Metric No. | Metric (Indicator) | Data Source Data Quality, Limitations and Improvements 
Number of public data sets included in Data Strategy Team Data Inventory and The data are validated with a crosswalk between Inventory entries and the listing of public 
ED Data Inventory and thus linked to the public ED Data Inventory at Department datasets, ensuring that the data described in the ED Data Inventory is 
Data.gov or ED.gov websites http://datainventory.ed.gov publicly available at the identified web address. 


5.1.B Number of states linking K-12 and State Longitudinal Data Systems (SLDS) Data are collected through monitoring of states with active SLDS grants. The data are 
postsecondary data with workforce data grant monitoring (monthly updates from limited to this population. While 47 states and territories have received these grants over 
states, annual performance reports, final time, by June 2015, there will be fewer than 25 states with active grants, which leads to 
performance reports, and site visits) incomplete and not up-to-date data from states that either have not receive grants or that 
do not have active grants. 


Number of states linking K-12 with early If the state is focusing on an early childhood data system, the state’s reporting might 

childhood data reflect only those development efforts, and not statewide P20W (preschool to college 
workforce) development efforts. There are additional sources for information about state 
data linkages in the field. For example the Data Quality Campaign (DQC) conducts 
surveys of state capacity to collect, store, link, and use data. Because the Department 
does not use the same definitions as DQC, our figures tend to be lower than theirs. For 
example, we require that a state possess the capacity to follow its own students in order 
for us to report that the state has the linkage in place. Similarly, we enable states to report 
on whether particular linkages are planned, in progress, or complete, and report that a 
state has a linkage when the state reports that the project is complete; DQC might give a 
state credit for an ‘in progress’ or pilot-stage linkage. 


A survey administered to the universe of states and territories would enable more 
systematically collect data about all states’ capacity for data linkages and data use. There 
is a concern, however, that if those data were to be used for public reporting, states might 
begin to overstate their capacities, particularly on data that are also publicly reported by 
organizations such as DQC. Currently, data from monitoring is used in an iterative, 
formative approach to program improvement; our technical assistance program is 
designed to support states’ efforts to improve their systems. This relies on states being 
honest about their own internal capacities. 


*Metric associated with an Agency Priority Goal. 
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MY (= carom (Lae L(er-1 ol g) Data Source Data Quality, Limitations and Improvements 


Average time to close “cases” (PTAC + Case Tracking System (CTS) The term “case” refers to requests for quick, informal responses to routine questions 

FPCO) Monthly metric reports related to student privacy. These requests are received via email, the Family Policy 
Compliance Office (FPCO) / Privacy Technical Assistance Center (PTAC) resource 
website, or by telephone and subsequently entered into the Case Tracking System (CTS). 


In contrast, “correspondence and complaints” refers to written complaints of alleged 
failures to comply with Family Educational Rights and Privacy Act (FERPA) / Protection of 
Pupil Rights Amendment (PPRA) filed with FPCO; requests for formal written 
guidance/interpretation laws administered by FPCO; and, to the reporting of instances of 
data breaches by educational agencies and institutions. These inquiries are logged into 
the Correspondence Control Manager (CCM) System, given a tracking control number, 
and assigned to FPCO staff. 


The preliminary data for this metric are reviewed at least weekly for verification. If 
anomalous data are identified in the periodic reviews or when anomalies are suspected, 
individual cases are examined individually to identify if they were properly closed or if their 
status was entered incorrectly. When appropriate, corrections are made. Staff responsible 
for entering data into the CTS will continue to be trained on policies and procedures. 


The monthly metric reports are scrutinized by the Director of FPCO, the Contracting 
Officer Representative for the PTAC contractors, and the Department's Chief Privacy 
Officer, to assure completeness and reliability of data and to recommend any 
improvements to the CTS or modifications to the standard operating procedures. The 
quarter entry represents the fiscal year to date average days to close as of the end of that 
quarter taken from the corresponding monthly report. 


*Percentage of select new (non- Forecast Report issued by the Office of the || In determining which discretionary grants are considered “evidence-based” (i.e., the 

continuation) discretionary grant dollars Chief Financial Officer and final Funding numerator when calculating the percentage), the Department includes all programs that 

that reward evidence Reports from relevant programs rewarded applicants with supporting evidence of promise or better (per the EDGAR 
evidence framework). This could be done through a competitive preference or absolute 
priority, an eligibility requirement, or a selection factor. Only the amounts of the grants 
awarded for those projects were counted. In determining what counts as discretionary 
funding (i.e., the denominator when calculating the percentage), the Department includes 
all programs for which the EDGAR evidence framework could conceivably work. In Fiscal 
Year 2015, the Department counted all discretionary grant programs except for those 
programs run through the Institute of Education Sciences (IES), which are already 
evidence-based and would not be candidates for the EDGAR evidence framework). 


Number of peer-reviewed, full-text To be counted as a full-text, peer-reviewed article in ERIC, the document had to be 
resources in the Education Resources indexed in ERIC and designated with a “peer-reviewed” and “full-text” flag, both of which 
Information Center (ERIC) are available on the public ERIC website 


The ERIC contractor uses specified quality assurance procedures. In addition, the IES 

program officer pays close attention to the metric in their review of deliverables. 
Number of reviewed studies in the What To be counted as a reviewed study, the study had to be listed in the What Works 
Works Clearinghouse (WWC) database Clearinghouse’s publicly available Database of Reviewed Studies. 


The What Works Clearinghouse contractors use specified quality assurance procedures. 
In addition, the IES program officers pay close attention to the metric in their review of 
deliverables. 


*Metric associated with an Agency Priority Goal. 
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WM (= catem (Lae L(ez-1Colg) Data Source Data Quality, Limitations and Improvements 


*Number of completed project Discretionary grant slate memoranda, Expectation is that it will take about five minutes to search the WWC database for new i3 

evaluations from grantees of select discretionary grant financial forecasts and studies each quarter. Also considering ways in which Institute of Education Sciences’ staff 

discretionary grant programs in a given reports from OCFO, and the What Works and Office of Innovation and Improvement staff can work closely to report these data 

fiscal year that meet What Works Clearinghouse quickly. 

Clearinghouse (WWC) Evidence 

Standards No formal verification or validation processes are in place or to check for anomalous data. 
So far, no anomalous data has been encountered. 


The Office will report out on studies that are funded by i3 that are in the WWC database 


and meet WWC Evidence Standards. Please note it is not implied that these studies all 
reflect positive results—in actuality, the WWC Evidence Standards only measure the rigor 
of impact evaluations and do not consider the actual outcomes reported. 


Percentage of schools in the country that || Education Superhighway (for baseline), The Department uses an external data source for this metric and relies on the external, 
have actual internet bandwidth speeds Consortium for School Networking third party’s verification and validation methodology. Based on the information provided 
of at least 100 Mbps (CoSN)/AASA (American Association of (http://cosn.org/about-cosn), the response rate for this survey may not be sufficient to 
School Administrators today known as ensure that the data are representative of all districts in the country. The Department is 
AASA, The School Superintendents exploring the feasibility of collecting data on access to and use of education technology 
Association) E-rate Infrastructure Survey from a representative sample of schools and districts across the country. In the 
meantime, we believe these data are the best currently available and provide useful 
information to gauge progress on this metric. 


*Metric associated with an Agency Priority Goal. 
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GOAL 6: U.S. DEPARTMENT of EDUCATION CAPACITY. Improve the organizational capacities of the Department to implement this strategic plan. 


WM (= Catem (Lae L(er-1 Col g) Data Source Data Quality, Limitations and Improvements 


Percent of selections made per job EDHires (Monster’s electronic hiring The data are entered into the EDHires system by customers, contractors, and staff, so 
opportunity announcement (JOA) management system) quality of data are only as reliable as the information entered and/or updated in the 
system. 


Data are reviewed by HR Specialists for relevance and completeness to ensure correct 
parameters and filters have been applied. 


If anomalies are identified, HR Specialists will compare contractor generated reports 
against ad hoc hiring reports generated from the system of record and other linked HR 
systems. Any questionable results would be brought to the contractor’s attention; in turn, 
OHR would work with the contractor to obtain clarity and/or resolve. 
EVS Employee Engagement Index Office of Personnel Management (OPM) Any questionable FEVS data points are brought to the attention of OPM. In turn, the OPM 
Federal Employee Viewpoint Survey point of contact responds to the inquiry. Further, if there are questions regarding the 
(FEVS) FEVS data, the Department works with the OPM point of contact to obtain clarity. 


The Engagement index score is calculated by OPM by first determining the percent 
positive for each of the 15 items in the sub-indices (i.e., Leaders Lead, Supervisors, and 
Intrinsic Work Experiences). Then the unrounded percent positive scores are averaged 
across the items in the index to get the index score. The overall index score is then 
rounded for reporting purposes. 
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name Metric (Indicator) Data Quality, Limitations and Improvements 


Time to Hire eI Personnel/Payroll System (FPPS) The Office of Management (OM) identified a gap between hiring procedures and the 90 
Datamart Day Hiring Model’s time to hire calculations which necessitated an adjustment in the way 
time to hire is calculated for some actions. OM found that recruitment work is often 
initiated prior to the receipt of an “official” action. While this is a proactive customer-centric 
approach, especially prevalent in ED’s noncompetitive hiring activities, it can sometimes 
result in a negative time to hire or a time to hire of zero—neither of which reflects a 
realistic or meaningful hiring lead time. 
To account for, track and document this upfront work, OM developed an adjustment 
mechanism to better gauge hiring lead time in these cases. For ease of comparison, the 
adjustment was applied to all FY 2016 hiring actions and resulted in a revised Q1 actual. 
To ensure clarity and consistency in the application of time to hire methodology moving 
forward, OM also clarified time to hire calculation protocols as follows: 
> — Individual time to hire: 

o — Time to hire for individual hiring actions is determined by calculating 
the number of days between the Request to Recruit (ROE) Date and 
the Verbal (tentative) Offer Date, as reflected in FPPS DataMart. 

. If time to hire results in a negative or zero, it is replaced 
with the value 1. 

. If time to hire cannot be determined (i.e., date not 
available), it is excluded from agency calculations. 

o Time to hire is not calculated for actions In-Progress; it is only 
calculated for Completed actions (i.e., estimated EOD established). 

> Agency time to hire: 

o The Department's time to hire is an annual rate determined by 
calculating the percent of hiring actions within a fiscal year completed 
within 90 days. 

Q1, Q2, and Q3 actuals are cumulative, measured from the 
beginning of the fiscal year. 

Q4 actuals represent the Department’s annual time to hire rate and 
are reported in the Annual Performance Report. 

Periodic snapshots of time to hire may be computed monthly, 
quarterly or by principal office to help gauge progress toward the 
annual target; however, care should be taken to ensure periodic 
updates are clearly identified as snapshots or progress indicators and 
not represented as the agency’s annual rate. 


Effective Communication Index Office of Personnel Management (OPM) Any questionable FEVS data points are brought to the attention of OPM. In turn, the OPM 
Federal Employee Viewpoint Survey point of contact responds to the inquiry. Further, if there are questions regarding the 
(FEVS) FEVS data, the Department works with the OPM point of contact to obtain clarity. 


This index score is calculated by the Department by averaging the percent positive scores 
from OPM FEVS questions 53, 58, and 64. 
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Percentage of A-133 Single Audits 
Overdue for resolution 


Compliance rate of contractor evaluation 
performance reports 


Overall average impact score of the 
Department's technical assistance in 
helping build State capacity to 
implement education reforms 


Office of the Chief Financial Officer's 
(OCFO) Audit Accountability & Resolution 
Tracking System (AARTS) 


Past Performance Information Retrieval 
System, www.ppirs.gov, “PPIRS 
Compliance Report” 


Annual Grantee Satisfaction Survey 
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| Metric No. | Metric (Indicator) Data Source Data Quality, Limitations and Improvements 


Calculations for this metric are determined by dividing the total number of audits that are 
overdue at the end of the Department's fiscal year by the total number of audits in the 
Department's inventory. 


Access rights to the AARTS database are managed by Office of the Chief Financial 
Officer (OCFO) and all users receive annual IT security training to help ensure data 
integrity. A Quality Control reviewer initiates a weekly upload of A-133 audit data to 
AARTS through a file submission directly from the Federal Audit Clearinghouse (FAC). An 
AARTS administrator (separate from the Quality Control reviewer) must verify the 
uploaded data with the actual audits. Data for individual programs are verified by Principal 
Offices (POs) through periodic review by Responsible Managers and Audit Liaison 
Officials. In addition, the specific data for this metric are verified by POs each month as 
part of monthly Dashboard reports. 


Data are validated by OCFO monthly. Staff work to reconcile data reported on the 
Dashboards with any discrepancies reported by the POs. 


Compliance rates of contractor performance evaluations are set by OMB and are 
calculated by use of a Government wide reporting tool available in the PPIRS 


(www.ppirs.gov). 


Office of the Chief Financial Officer (OCFO) and Federal Student Aid (FSA) Acquisitions 
staff actively monitor the PPIRS report to ensure that each contractor performance 


evaluation reflected on that report should appear on that report, and to rectify any errors 
on the report. 

The contractor has a quality control system and the results are also provided to all of the 
programs that participate in the survey to identify issues/anomalies with the data. 


Program staff report anomalies to the contractor to correct. 


The data comes from the Annual Grantee Satisfaction Survey Report (See 
https://www2.ed.gov/about/reports/annual/gss/index.html). The report is produced by a 
contractor; the contract is managed by OCFO. 


One major issue with the Grantee Satisfaction Survey is the timing of the survey. The 
survey is administered in late spring/early summer when many staff are out of the office, 
which suppresses the response rate. Also, some grant programs do a better job than 
others of promoting the survey and showing how the survey results have been used to 
make improvements. We have been told that the survey is administered late each year for 
the following reasons: 

--The Interagency Agreement can’t be put into clearance until the budget is approved 
each year; 

--Getting the Interagency Agreement through Departmental clearances takes several 
months; program staff are asked to review their customized survey questions and submit 
their grantee contact lists at the busiest time of the year which makes it difficult to turn 
them around quickly. 


124 


APPENDICES 
DATA VALIDITY AND VERIFICATION 


Number of ED IT security incidents Cyber Security Assessment and Office of the Chief Information Officer (OCIO) has standard operating procedures (SOP) 
Management (CSAM) and RSA Security to verify and validate the data: 1) CSAM SOP, 2) Incident Response SOP, and 3) SecOps 
Operations management (SecOps) SOP. This process is executed on a weekly basis. Further, the data is pulled from the 
Department's authoritative source (system of record) for Incident Response and Tracking. 
In addition, SOPs enforce the use of this capability for entering, tracking, and managing 
all incident related activity. The data is also audited on a routine basis to ensure 
consistency and accuracy. 


EVS Results-Based Performance Office of Personnel Management (OPM) Data verification and validation process is integrated into the OPM FEVS survey results 
Culture Index Federal Employee Viewpoint Survey validation process. Any questionable FEVS results would be brought to the attention of 
EVS Leadership and Knowledge (FEVS) OPM; the Department would then work with the OPM point of contact to obtain clarity. 
Management Index 


Data verification and validation process is integrated into the OPM FEVS survey results 
validation process. The Results-Oriented Performance Culture Index score is calculated 
by OPM by first determining the percent positive for each of the 13 items in the index. 
Then the unrounded percent positive scores are averaged across all index items to get 
the index score. The index score is then rounded for reporting purposes. 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 125 


APPENDICES 
DATA VALIDITY AND VERIFICATION 


WM C= Catom (Lae L(er-1 ol g) Data Source Data Quality, Limitations and Improvements 


Total usable square footage Department's Master Space Management The Department reconciles its rent bills per building on a monthly basis. Every six months 
Plan the usable square footage is verified with GSA. 


Because usable square footage relates directly to rent costs, the Department uses the 
same data verification and validation procedures. Specifically, the data are collected 
directly from Occupancy Agreements and rent bills per building. 


The data are derived from historic examples and relevant experience. Department 
leadership has agreed to a set of assumptions by which the data are based. Leadership 
has reached out to subject matter experts to broaden the scope of the data set, and lower 
risks of missing contingencies that may affect the data. At each step, the data are 
reviewed independently to double check the work of each team member and provide 
quality control. These processes help ensure the data’s completeness and reliability. 


For the baseline data, the Department made the following assumptions: 

1) All leased buildings: 2% is applied for anticipation of CPI (Consumer Price Index) 
annual increases on the anniversary date of the active lease/occupancy agreement (OA); 
and 2.5% is applied for anticipation of annual tax increases. 

2) All federal buildings: 2.5% is applied for operating cost escalations on the anniversary 
date of the active OA. 

3) 20% is applied to all federal buildings after an OA has expired and a new OA is 
unavailable. (Projected increase on the appraisal.) 

4) 40% is applied to all leased buildings after an OA has expired and a new OA is 
unavailable. (Projected increase on the market rent.) 

5) If a new OA is unavailable, 3 months’ early rent is applied to all buildings that are 
relocating due to possible Department delays. Example: Changes made to the designs 
after Scope of Work (SOW) is completed. 

6) 3 months’ late rent is applied to all buildings that are relocating due to possible 
Department delays. For example, delays in returning space back to a rentable condition. 


The Department reconciles its rent bills per building on a monthly basis. 


Rent cost Department's Master Space Management Data are collected directly from OAs and rent bills per building. The actual rent may vary 
Plan significantly if the Department relocates to a new leased building and/or signs short lease 
extensions. The Department is leveraging the examples and experience of the mobility 
labs and building consolidations programs. 


The Department reconciles its rent bills per building on a monthly basis. Every six 
months, leadership will re-evaluate the data, the assumptions on which it is based, and 
incorporate actual costs and project schedules. These steps will become part of our 
quality assurance program and procedures. Leadership looks to improve completeness, 
reliability, and quality of the data at these milestones. 


Total number of the Department’s 2016 external (public-facing) metrics (indicators): |52 
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The Government Performance and Results Act Modernization Act of 2010 requires 


agencies to “identify performance goals changed or dropped since publication of the 
Annual Performance Plan, if such changes were approved by OMB, and the reasons 
for the changes.”92 


2 OMB Circular A-11, Preparation, Submission, and Execution of the Budget (2015), Part 6, Section 210.22 — Content for Strategic 
Plans, Annual Performance Plans, Annual Performance Reports 
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FY 2016 EXTERNAL (PUBLIC-FACING) METRICS BEING REMOVED and NEW METRICS BEING ADDED FOR 2017 


GOAL 1: POSTSECONDARY EDUCATION, CAREER AND TECHNICAL EDUCATION, AND ADULT EDUCATION. Increase college access, 


Flacelccl-leliliavemelerliinam-lacemeelialel(-d(elamey mln elaoniial-mexeiax-\xelaler-larm-e (ele 1dtelam-laremibiclelal-m-r-laallat-ae)e) elelaqelalia(-cm cola celUldace-lalem-LelU Lae 


EXTERNAL 
METRIC NO. 


NEW METRIC DATA 
METRIC (INDICATOR) SOURCE 
Enrollment in IHEs below the minimum The Department has minimal impact No replacement metric is being NA 

earnings threshold on this metric and the rate did not proposed at this time. 
change positively in FY 2016. 
1.1.C. Percentage of first-time FAFSA filers among Federal Student Aid (FSA) reports this | Number of first-time FAFSA filers | FSA’s Central Processing 
high school seniors metric in their Annual Performance among high school seniors System 

Report with ranges as targets. The 
Department is moving away from 

using ranges for any of its metrics. The 
new metric will focus on the number of 
first-time FAFSA filers among high 
school seniors, which is 
complementary to what FSA is already 
reporting. Further, the new metric 
highlights the actual impact of FSA’s 
reach by reflecting the actual number 
of students completing the FAFSA 
versus reporting just the percentage 
year-over-year. 


NEW METRICS FOR 2017 


BRIEF SUMMARY FOR REMOVAL 
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GOAL 2: ELEMENTARY AND SECONDARY EDUCATION. Improve the elementary and secondary education system’s ability to consistently 


deliver excellent instruction aligned with rigorous academic standards while providing effective support services to close achievement and 
opportunity gaps, and ensure all students graduate high school college- and career-ready 


EXTERNAL 
METRIC NO 


BRIEF SUMMARY FOR 


REMOVAL 


In December 2015, the President 
signed into law the Every Student 
Succeeds Act (ESSA), which 
reauthorizes the Elementary and 
Secondary Education Act of 1965 
(ESEA). Under the ESEA as amended 
by the ESSA, the Secretary of 
Education is prohibited from, among 
other prohibitions, prescribing “any 
aspect or parameter of a teacher, 
principal, or other school leader 
evaluation system within a State or 
local educational agency” and 
“indicators or specific measures of 
teacher, principal, or other school 
leader effectiveness or quality.” The 
Department has no new data on the 
number of states implementing 
teacher and leader evaluation and 
support systems consistent with this 
performance metric and no way to 
verify the data reported. 


NEW METRIC DATA 
SOURCE 


METRIC (INDICATOR) NEW METRIC(s) FOR 2017 


Number of states that have fully implemented 
teacher and principal evaluation and support 
systems that consider multiple measures of 
effectiveness, with student growth as a 
significant factor 


No replacement metric is being 
proposed at this time. 


Percentage of high school and middle school 
teachers who teach STEM as their main 
assignment who hold a corresponding 
undergraduate degree 


Retiring this metric due to the data set 
only being available every four years 
and therefore has not been helpful to 
inform STEM policy efforts. In addition, 
degree held is just one indicator of 


STEM content knowledge and doesn’t 
apply for things like elementary school 
teaching. The data set was also only 
applicable for math and science and 
didn’t capture engineering or computer 
science, which have been important 
STEM disciplines as well. 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 


No replacement metric is being 
proposed at this time. 
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GOAL 3: EARLY LEARNING. Improve the health, social-emotional, and cognitive outcomes for all children from birth through 3rd grade, so 


that all children, particularly those with high needs, are on track for graduating from high school college- and career-ready. 


EXTERNAL BRIEF SUMMARY FOR NEW METRIC DATA 
aac: METRIC (INDICATOR) A EMOUAL NEW METRIC(s) FOR 2017 paar 


NA 


GOAL 4: EQUITY. Increase educational opportunities for underserved students and reduce discrimination so that all students are well- 
positioned to succeed. 


Percentage of proactive civil rights 
investigations launched annually that address 
areas of concentration in civil rights 
enforcement 


To ensure educational 
institution’s awareness of and 
compliance with federal civil 
rights obligations, the new metric 
focuses on increasing the 
average number of cases 
resolved, per investigative staff 
member. 
4.2.B. Percentage of proactive civil rights This metric is too narrow to capture To increase awareness and OCR’s Case Management 
investigations resolved annually that address the breadth of the Office for Civil understanding by students and System (CMS) and website 
areas of concentration in civil rights Rights’ (OCR) work. A new metric is their families of their rights, and 
enforcement being proposed. schools, colleges and universities 
of their obligations under federal 
civil rights, the new metric 
focuses on increasing the 
number of technical assistance 
presentations and press 
releases on OCR’s work. 


This metric is too narrow to capture 
the breadth of the Office for Civil 
Rights’ (OCR) work. A new metric is 
being proposed. 


OCR’s Case Management 
System (CMS) and human 
resources dashboard 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 130 


APPENDICES 


EXTERNAL 
METRIC NO 
1.1.B. 


BRIEF SUMMARY FOR 


METRIC (INDICATOR) REMOVAL 


Web traffic to the College Scorecard 


Number of states linking K-12 with early 
childhood data 


Requesting to replace this metric with 
a new metric related to the number of 
states actively using data systems to 
support and inform improvements, 

which is a more meaningful and useful 
metric. 


plan. 


EXTERNAL 


BRIEF SUMMARY FOR 


METRIC (INDICATOR) 


6.2.A 


Percentage of A-133 Single Audits Overdue 
for resolution 


OCFO/Financial Improvement 
Operations is proposing this change 
because the new measure will 
potentially be a more accurate 
measure of the efforts of both program 
office (grants administration) staff and 
staff involved in resolution of Single 
Audits to positively influence lasting 
program improvement. Lack of any 
audit findings is a reflection of effective 
administration by grant recipients and 
improvements over time will validate 
the Department's overall efforts to 
partner with them to effect positive 
change in their operations. 


Total number of the Department’s 2016 external metrics being removed FY-end: 8" 


Total number of new external metrics being added in 2017: |5 


* Please note, the total number of metrics removed in FY 2015 was 11 versus the 12 reported in the 2015 APR. 
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GOAL 5: CONTINUOUS IMPROVEMENT OF THE U.S. EDUCATION SYSTEM. Enhance the education system’s ability to continuously improve 


through better and more widespread use of data, research and evaluation, evidence, transparency, innovation, and technology. 


GOAL 6: U.S. DEPARTMENT of EDUCATION CAPACITY. Improve the organizational capacities of the Department to implement this strategic 


CHANGED PERFORMANCE GOALS 


NEW METRIC DATA 


NEW METRIC(s) FOR 2017 SOURCE 


This performance metric is NA 
transitioning from Goal 1 to Goal 
5. 


Number of states actively using 
data systems to support and 
inform improvements 


Grant monitoring 


NEW METRIC(s) FOR 2017 | “EW Pree eat 


Percentage of Department grant 
recipients without any Single 
Audit findings 


Federal Audit Clearinghouse 
(FAC) 
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Most of the Department’s 100-plus programs are funded through discretionary appropriation 
acts enacted each fiscal year. However, there are many education programs—some of them 
large—that are funded directly through their authorizing statutes. For many budgeting purposes, 
these programs are classified as mandatory. For the purposes of this document, resources by 
goal are discretionary funds only. Mandatory programs that contribute to each goal are listed 
below. 


Goal 1: Postsecondary Education, Career and Technical Education, 
and Adult Education 


Other discretionary Goal 1 programs/activities include the following: 


POC Account Objective Program 
FSA SFA 1 Federal supplemental educational opportunity grants 
FSA SFA 1 Federal work-study 
OCTAE HE NA _ Tribally controlled postsecondary career and technical institutions 
OESE ED NA _ Special programs for migrant students 
OPE HE 1,2,3 Aid for institutional development: Strengthening institutions 
OPE HE 1,2,3 Aid for institutional development: Strengthening tribally controlled colleges and 
universities 
OPE HE 1,3. Child care access means parents in school 
OPE HE College Housing and Academic Facilities Loans Program Account: Federal 
administration 
OPE HE 1,2,3 Developing Hispanic-serving institutions 
OPE HE 1. Gaining early awareness and readiness for undergraduate programs (GEAR UP) 
OPE HE 3 Graduate assistance in areas of national need 
OPE HE Historically Black College and University Capital Financing Program Account: Federal 
administration 
OPE HE Historically Black College and University Capital Financing Program Account: Loan 
subsidies 
OPE HE Howard University Hospital 
OPE HE 2,3 Howard University: General support 
OPE HE 2 International education and foreign language studies: Domestic programs 
OPE HE 2 International education and foreign language studies: Overseas programs 
OPE HE 1,2,3. Minority science and engineering improvement 
OPE HE 1,2 Model transition programs for students with intellectual disabilities into higher 
education 
OPE HE 1,2,3 Promoting postbaccalaureate opportunities for Hispanic Americans 
OPE HE 1,2,3. Strengthening Alaska Native and Native Hawaiian-serving institutions 
OPE HE 1,2,3 Strengthening Asian American- and Native American Pacific Islander-serving 
institutions 
OPE HE 1,2,3 Strengthening HBCUs 
OPE HE 1,2,3 Strengthening historically Black graduate institutions 
OPE HE 1,2,3. Strengthening Native American-serving nontribal institutions 
OPE HE 1,2,3. Strengthening predominantly Black institutions 
OSERS GU 2,3 Gallaudet University 
OSERS NTID 2,3 National Technical Institute for the Deaf 


NA = Not applicable. 
POC = Principal Operating Component. 


Mandatory programs supporting Goal 1 include: 


POC’ Account Objective Program 

FSA FDSL 1 Federal Direct Student Loans Program Account: Downward modification of existing 
loans (non-add) 

FSA FDSL 1 Federal Direct Student Loans Program Account: Downward re-estimate of existing 


loans (non-add) 
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POC 
FSA 


FSA 


FSA 
FSA 
FSA 
FSA 
FSA 
FSA 


FSA 
FSA 
FSA 
FSA 
FSA 


FSA 
FSA 
FSA 
FSA 
FSA 
FSA 
FSA 
FSA 
FSA 
OPE 


OPE 
OPE 
OPE 
OPE 


OPE 
OPE 
OPE 


OPE 
OPE 
OPE 
OSERS 
OSERS 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 
SFA 


Account 
FDSL 


FDSL 


FDSL 
FDSL 
FDSL 
FDSL 
FFEL 
FFEL 


FFEL 
FFEL 
FFEL 
FFEL 
FFEL 


FPL 
HEAL 
SFA 
SFA 
SFA 
TEACH 
TEACH 
TEACH 
TEACH 
HE 


HE 
HE 
HE 
HE 


HE 
HE 
HE 


HE 
HE 
HE 
REHAB 
REHAB 


Objective 
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ADDITIONAL PROGRAMS BY GOAL 


Program 


Federal Direct Student Loans Program Account: Net modification of existing loans 
(non-add) 

Federal Direct Student Loans Program Account: Net re-estimate of existing loans 
(non-add) 

Federal Direct Student Loans Program Account: New loan subsidy 


Federal Direct Student Loans Program Account: New net loan subsidy (non-add) 
Federal Direct Student Loans Program Account: Upward modification 

Federal Direct Student Loans Program Account: Upward re-estimate of existing loans 
Federal Family Education Loans Liquidating Account: Pre-1992 student loans 


Federal Family Education Loans Program Account: Downward modification of existing 
loans (non-add) 

Federal Family Education Loans Program Account: Downward re-estimate of existing 
loans (non-add) 

Federal Family Education Loans Program Account: Net modification of existing loans 
(non-add) 

Federal Family Education Loans Program Account: Net re-estimate of existing loans 
(non-add) 

Federal Family Education Loans Program Account: Upward modification of existing 
loans 

Federal Family Education Loans Program Account: Upward re-estimate of existing 
loans 

Federal Perkins Loan Program 


Health Education Assistance Loans Liquidating Account 

Federal Pell grants: Mandatory 

Federal Pell grants: Mandatory funding for discretionary program costs 
lraq and Afghanistan service grants 

TEACH Grants: Downward re-estimate of existing loans (non-add) 
TEACH Grants: Net re-estimate of existing loans (non-add) 

TEACH Grants: New loan subsidy 

TEACH Grants: Upward re-estimate of existing loans 


Aid for institutional development: Mandatory strengthening tribally controlled colleges 
and universities 
College housing and academic facilities loans liquidating account 


College housing loans liquidating account 
Higher education facilities loans liquidating account 


Historically Black college and university capital financing program account: 
Re-estimate of existing loan subsidies 
Mandatory developing HSI STEM and articulation programs 


Mandatory strengthening Alaska Native and Native Hawaiian-serving institutions 


Mandatory strengthening Asian American- and Native American Pacific Islander- 
serving institutions 
Mandatory strengthening HBCUs 


Mandatory strengthening Native American-serving nontribal institutions 
Mandatory strengthening predominantly Black institutions 
Vocational rehabilitation, Grants to Indians 

Vocational rehabilitation, state grants 

CHAFL downward re-estimate of loan subsidies 

FDSL downward modification/negative loan subsidies 

FDSL downward re-estimate of loan subsidies 

FFEL downward modification/negative loan subsidies 

FFEL downward re-estimate of loan subsidies 

General receipts, not otherwise specified 

HBCU capital financing downward re-estimate of loan subsidies 
HEAL downward re-estimate of loan subsidies 

Perkins Institutional fund recall (mandatory) 

Perkins loan negative loan subsidies 

Perkins loan repayments 

Proprietary receipts 

Student financial assistance debt collection 

TEACH downward modification/negative loan subsidies 
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Goal 2: Elementary and Secondary Education 


Other discretionary Goal 2 programs/activities include the following: 


POC Account Objective Program 


OESE ED NA Comprehensive literacy development grants 
OESE ED NA _ Innovative approaches to literacy 
OESE ED NA _ State agency programs: Neglected and delinquent 
OESE 1&1 NA Advanced placement 
OESE 1&1 NA _ Javits gifted and talented education 
OESE 1&1 2 Supporting effective educator development (SEED) 
OESE IA 3. Impact Aid: Construction 
OESE IA 3. Impact Aid: Facilities maintenance 
OESE IA 2,3 Impact Aid: Payments for federal property 
OESE IA 2,3 Impact Aid: Payments for federally connected children: Basic support payments 
OESE IA 2,3 Impact Aid: Payments for federally connected children: Payments for children with 
disabilities 
OESE SIP NA _ Education for homeless children and youths 
OESE SIP 2,3,4 Rural education 
OESE SIP Supplemental education grants 
OESE SSCE 3 Elementary and secondary school counseling 
OESE SSCE 3 Physical education program 
OESE SSCE 3 School safety national activities 
Oll 1&1 NA American history and civics education 
Oll 1&1 NA _ Arts in education 
Oll 1&l 3 Non-cognitive skills initiative 
Oll 1&1 2 School leader recruitment and support 
Oll 1&1 2 Teacher quality partnership 
Oll SSCE 1,2,3 Full-service community schools 
OSERS APBH 1. American Printing House for the Blind 
OSERS SE Education technology, media, and materials 
OSERS SE 3 Parent information centers 
OSERS SE 2 Personnel preparation 
OSERS SE 2 State personnel development 


NA = Not applicable. 
POC= Principal Operating Component. 


Goal 3: Early Learning 


Other discretionary Goal 3 programs/activities include the following: 


POC’ Account Objective Program 
Oll 1&1 NA _ Ready to learn programming 
NA = Not applicable. 
POC = Principal Operating Component. 


Goal 4: Equity 
No additional programs. 
Goal 5: Continuous Improvement of the U.S. Education System 


Other discretionary Goal 5 programs/activities include the following: 


POC Account Objective Program 
IES IES 3 National Assessment Governing Board 
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POC 
IES 
OCTAE 
OCTAE 
OESE 
OESE 
OSERS 


Account 
IES 
CTAE 
CTAE 


SE 


Objective Program 


ADDITIONAL PROGRAMS BY GOAL 


3 Special education studies and evaluations 


1,3 


3 Career and technical national programs 
2 Indian Education: National activities 


1,3,4 
1,2,3,4 


POC = Principal Operating Component. 


Comprehensive centers 
Technical assistance and dissemination 


Goal 6: U.S. Department of Education Capacity 


Mandatory programs supporting Goal 6 include: 


POC Account 


Objective Program 


Contributions 


POC = Principal Operating Component. 


Other 


Programs/activities supporting other performance metrics include: 


(Dollars in millions) 


Adult education national leadership activities 


FY 2018 
FY 2016 FY 2017 President's 
POC Account Program Appropriation || Annualized CR’ Budget 
OSERS REHAB _ Client assistance state grants 13 13 13 
OSERS REHAB _ Demonstration and training programs 6 6 6 
Helen Keller National Center for Deaf-Blind 
OSERS REHAB _ Youths and Adults 10 10 10 
Independent living services for older blind 
OSERS REHAB individuals 33 33 33 
OSERS REHAB _ Protection and advocacy of individual rights 18 18 18 
OSERS REHAB _ Supported employment state grants 28 27 -- 
OSERS REHAB _ VR Training 30 30 30 
College housing and academic facilities 
loans program account: Re-estimate of 
existing loan subsidies 0 0 a 
TOTAL, OTHER 138 138 110 


POC = Principal Operating Component. 
CR = Continuing Resolution. 
Note: Detail may not add to total due to rounding. 


* A full-year 2017 appropriation was not enacted at the time the FY 2018 Budget was prepared; therefore, the Budget is built off of 
the Further Continuing Appropriations Act, 2017 (P.L. 114-254). The amounts included for 2017 reflect the annualized level 
provided by the continuing resolution. 
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Appendix D: Summary of Performance Evaluations Released 
During FY 2016-17 and Expected During FY 2017-18 


For a complete list of program evaluations and studies from the Office of Planning, Evaluation, 
and Policy Development, please visit 


http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html. For a complete list of evaluation 
studies of the National Center for Education Evaluation and Regional Assistance, please visit 


http://ies.ed.gov/ncee/projects/evaluation/index.asp. 
Evaluation Reports From FY 2016-17 


National Center for Education Evaluation and Regional Assistance 


Literacy 


Summary of Research Generated by Striving Readers on the Effectiveness of 
Interventions for Struggling Adolescent Readers 


1. Study Purpose: Striving Readers was a discretionary grant program focused on raising 
reading achievement of middle school and high school students through intensive 
interventions for struggling readers and enhancing the quality of literacy instruction across 
the curriculum. The 2006 and 2009 cohorts of grantees were required to conduct rigorous, 
experimental evaluations. The Institute of Education Sciences (IES) provided technical 
assistance to support grantees in conducting evaluations that would meet What Works 
Clearinghouse (WWC) standards. This study also conducted a systematic review of the 
17 evaluations of 10 different interventions funded by Striving Readers across the 16 grants. 


2. Key Question Addressed: 


e What do the evaluations funded by the Striving Readers program tell us about effective 
interventions for improving the achievement of struggling adolescent readers? 


3. Design: The study is descriptive; it provided technical assistance to the local evaluations of 
Striving Readers grantees and also reviewed the completed evaluations against the WWC 
evidence standards. The report summarizes the results of this systematic review of the 
evaluations. 


4. Estimated or Actual Completion Date: The final report was released in October 2015. 
5. Key Findings: 


e Fifteen of the 17 evaluations of the interventions funded by the Striving Readers grant 
program met WWC evidence standards with or without reservations. 


e Based on findings from the 15 evaluations that met WWC evidence standards with or 
without reservations, 4 of the 10 interventions funded by the Striving Readers grant 
program had positive, potentially positive, or mixed effects on reading achievement. 


e The Striving Readers studies not only expand the evidence base on effective reading 
interventions for adolescents, but also increase the number of high-quality effectiveness 
studies reviewed by the WWC on this topic. Three of the four interventions that had 
positive, potentially positive, or mixed effects on reading achievement had not previously 
been reviewed by the WWC. 
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6. Link to Additional Information: http://ies.ed.gov/ncee/pubs/20164001/ 
School Choice 


Evaluation of the DC Opportunity Scholarship Program: Descriptive Reports Based on 
Parent and Principal Surveys 


1. Study Purpose: The April 2011 Scholarships and Opportunities for Results Act (SOAR Act) 
provided for a five-year continuation of a school choice program for low-income residents of 
Washington, DC. The program, still titled the Opportunity Scholarship Program (OSP), now 
provides annual scholarships of $8,000 (for grades K-8) or $12,000 (for grades 9-12) to 
enable low-income students to attend private schools in DC in lieu of the public schools 
already available to them. The new law also mandated another independent, rigorous 
evaluation of the effectiveness of the program. 


2. Key Questions Addressed: 


e What is the impact of the OSP on student academic achievement and other measures of 
student success, overall and for subgroups of students identified in the statute as high 
priority? 


e What effect does the program have on student and parent perceptions of school safety 
and satisfaction, and on parents’ involvement in education of their children? 


e Does the program change students’ instructional environments and opportunities? 


3. Design: The evaluation will primarily compare outcomes of approximately 1,800 student 
applicants randomly assigned by lottery to either receive a scholarship or not receive a 
scholarship. Lotteries of program applicants were conducted in spring 2012 (cohort 1), 
spring 2013 (cohort 2), and spring 2014 (cohort 3). Data will be collected for three follow-up 
years for each of the cohorts and, for students in both the scholarship and nonscholarship 
groups. The contractor will administer academic assessments, and conduct student, parent, 
and principal surveys each spring (spring 2013-17). Prior to the first impact report, 
descriptive reports based on application forms parents filled out when they applied to the 
OSP, principal surveys, and extant data will be released. 


4. Estimated or Actual Completion Date: The second and third descriptive reports—focused on 
select findings from parent and principal surveys—were released in 2016. The first impact 
report is scheduled for completion in early 2017. 


5. Key Findings from the Second Report: 


e The majority of parents seeking a private school voucher for their child were satisfied 
with the school their child was attending at the time of application. Fifty-seven percent of 
applicants’ parents gave their child’s current school at time of application a grade of ‘A’ 
or ‘B’, while 14 percent gave their child’s school a ‘D’ or ‘F.’ 


e Most applicants’ parents (49 percent) chose academic quality as their top priority when 
selecting a new school. The next most commonly selected top priority was school safety 
(selected by 18 percent of applicants’ parents). 


e Among the 49 percent of parents that said academic quality was their top priority for a 
new school, over 85 percent were satisfied with the academic quality of the school their 
child was attending when they applied for a scholarship. This suggests that families do 
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not always apply for private school vouchers because they are dissatisfied with key 
elements of the schools their children are already attending. 


Key Findings from the Third Report: Spring 2014 survey responses from principals at DC 
public schools—both traditional and charter—and private schools that participated in the 
OSP indicate that private schools participating in the OSP: 


e Were more likely than public schools to report that student behavior, student motivation, 
parental support for learning, and the instructional skills of teachers were excellent or 
very good; 


e Were less likely than public schools to suspend students, use metal detectors, and have 
on-site security personnel; 

e Taught reading and math for fewer minutes a week across grade levels than public 
schools; and 

e Were less likely than public school to have instructional programs for non-English 
speakers and students with learning disabilities. 


Links to Additional Information: https://ies.ed.gov/ncee/pubs/20164003/, 
https://ies.ed.gov/ncee/pubs/20164007/ 


Students with Disabilities 


Evaluation of Response to Intervention Practices for Elementary School Reading 


1 


Study Purpose: The focus of this study was on the implementation and effects of Response 
to Intervention (Rtl) practices for elementary school reading. Rtl is a multistep approach to 
providing early and more intensive intervention and monitoring within the general education 
setting. In principle, Rt! begins with research-based instruction provided to students in the 
general education classroom, followed by screening of all students to identify those who 
may need systematic progress monitoring, intervention, or support. Students who are not 
responding to the general education curriculum and instruction are provided with 
increasingly intense interventions through a “tiered” system, and they are frequently 
monitored to assess their progress and inform the choice of future interventions. This 
evaluation occurred under the National Assessment of the Individuals with Disabilities 
Education Act (IDEA), which permits districts to use Rtl to inform decisions regarding a 
child’s eligibility for special education. 


Key Questions Addressed: 


e How do Atl practices for early grade reading vary across schools? 


e How do schools experienced with Rtl vary the intensity of reading instruction for children 
at different reading levels? 


e What are the effects on grade 1-3 reading achievement of providing reading 
interventions to children who fell just below school-determined standards for each grade 
on screening tests? 


Design: The evaluation relied on a combination of descriptive data collection from school 
staff and regression discontinuity methods to address the research questions, and focused 
on practices in place during the 2011-12 school year in a sample of 146 elementary schools 
from 13 states. 
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4. Actual Completion Date: The study report was released in November 2015. 


5. Key Findings: 


Full implementation of the Rtl framework in grade 1-3 reading was reported by 
86 percent of the experienced elementary schools sampled for in-depth study, compared 
with 56 percent of a random sample of elementary schools from the same 13 states. 


The schools sampled for in-depth study of their Rtl practices provided more intense 
small-group instruction and more reading intervention services for groups of students 
reading below grade level than for groups of students reading at or above grade level. 
However, these schools’ implementation of Rtl differed in some ways from descriptions 
in the prior literature: 


e In grade 1, 45 percent of schools provided intervention services to some groups of 
students at all reading levels, rather than only for reading groups below grade level. 


e In grade 1, 67 percent of schools provided at least some reading intervention during 
the core reading block, rather than only in addition to the core. 


Grade 1 students who scored just below school-determined benchmarks on fall 
screening tests, and who were assigned to reading interventions, had lower spring 
reading scores than similar students in the same schools who were not assigned to 
those interventions. Unexplored, but plausible, factors that may be related to these 
negative effects include incorrect identification of students for intervention and a 
mismatch between the interventions provided and these students’ instructional needs. 
No statistically significant effects of assignment to reading interventions were found for 
students in Grades 2 or 3. 


6. Link to Additional Information: http://ies.ed.qov/ncee/pubs/20164000/ 


Teacher Effectiveness 


Evaluation of the Teacher Incentive Fund: Implementation and Impacts of Pay-for- 
Performance After Three Years 


1. 


Study Purpose: The Teacher Incentive Fund (TIF) was renamed the Teacher and School 
Leader Incentive Program with reauthorization of the Elementary and Secondary Education 
Act (ESEA) in the Every Student Succeeds Act (ESSA). The purpose of the TIF program is 
to develop and implement performance-based compensation systems (PBCSs) for teachers, 
principals, and other personnel in high-need schools. Research indicates that high-quality 
teachers are critical to raising student achievement in low-performing schools, but schools 
most in need often have difficulty in attracting and retaining high-quality teachers. 
Performance pay is a policy promoted by the TIF program to improve the quality of teachers 
in high-need schools. This evaluation studies performance pay that provides substantial and 
differentiated bonus pay to high-performing teachers in low-performing schools with high- 
need students. 


2. Key Questions Addressed: 


What are the characteristics of all TIF districts and their performance-based 
compensation systems? What implementation experiences and challenges did TIF 
districts encounter? 
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e How do teachers and principals in schools that did or did not offer pay-for-performance 
bonuses compare on key dimensions, including their understanding of TIF program 
features, exposure to TIF activities, allocation of time, and attitudes toward teaching and 
the TIF program? 


e What is the impact of pay-for-performance bonuses on students’ achievement on state 
assessments of math and reading? 


e How do pay-for-performance bonuses affect educator mobility, including whether 
mobility differs by educator effectiveness? 


3. Design: Study schools were randomly assigned within a grant to either implement all 
components of the PBCS or the PBCS with a 1 percent across-the-board bonus in place of 
the differentiated effectiveness incentive component of the PBCS. Data collection will 
include a grantee survey, a survey of teachers and principals, teacher and principal school 
assignment records, student record information (Such as student demographics and student 
test scores), and grantee interviews to document implementation information, as well as to 
conduct impact analyses. 


4. Estimated or Actual Completion Date: A report on the implementation and early impacts 
after one year was released in September 2014. The second report on the ongoing 
implementation and impacts after two years was released in September 2015. The third 
report on implementation and impacts after three years was released in August 2016. A 
fourth and final report is scheduled for release in the fall of 2017. 


5. Key Findings: The main findings among all TIF districts with 2010 awards are: 


e Similar to the previous two years, most districts (88 percent) implemented at least three 
of the four required program components for teachers. 


e By the third year, reported implementation challenges decreased, with no more than 
one-fifth of TIF districts reporting any major challenges. 


For the subset of 10 districts that agreed to participate in a random assignment study, 
key findings on the effect of pay-for-performance on educators include the following: 


e After three years of TIF implementation, average student achievement remained 1 to 
2 percentile points higher in schools that offered pay-for-performance bonuses than 
in schools that did not. This difference was equivalent to a gain of about four 
additional weeks of learning. 


e Atleast half of the evaluation districts each year met the grant guidance for awarding 
differentiated performance bonuses for teachers. However, in each year, no more 
than 2 of the 10 districts awarded bonuses for teachers that were substantial or 
challenging to earn. 


e Teachers’ understanding of performance measures continued to improve, but only about 
60 percent of teachers correctly reported being eligible for a performance bonus. In 
addition, teachers believed that the maximum bonus they could earn was no more than 
two-fifths the size of the actual maximum bonus that districts awarded, a finding similar 
to previous years. 


6. Link to Additional Information: http://ies.ed.gov/ncee/pubs/20164004/ 
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Focusing on Mathematical Knowledge: The Impact of Content-Intensive Teacher 
Professional Development 


Ts 


Study Purpose: Title Il, Part A of the current ESEA provides monetary resources to improve 
teacher quality through professional development. Currently there is relatively little rigorous 
empirical evidence on the components of an effective professional development program. At 
the same time, there has been renewed policy interest in fostering a globally competitive 
STEM workforce, which will require teachers to effectively impart mathematics to their 
students. This study will present implementation and impact findings from a random 
assignment evaluation of an intensive math professional development intervention focused 
on enhancing teacher content knowledge and integrating this knowledge into the classroom. 


Key Questions Addressed: 


e What is the average impact on (a) teachers’ content knowledge, (b) teachers’ classroom 
practices, and (c) student achievement of offering a specialized professional 
development intervention (in this case, Intel Math, Mathematics Learning Communities, 
and Video Feedback Cycles) relative to “business-as-usual” professional development? 


e How is the professional development intervention implemented? 


Design: Approximately 200 volunteer fourth-grade math teachers from six districts around 
the country participated in this study during the 2013-14 school year. The intervention has 
three components: i) Intel Math (80 hours largely focused on developing teachers’ 
understanding of K-8 mathematics), ii) Mathematics Learning Communities (which offers 
teachers opportunities to connect their learning to the classroom through a professional 
learning community aligned to Intel Math that focuses on student thinking), iii) Video 
Feedback Cycles (which provide teachers with individualized feedback three different times 
based on videotaped lessons of their classroom teaching reviewed by certified analysts 
using the Mathematical Quality of Instruction [MQI] rubric). Within each participating school, 
fourth-grade teachers were randomly assigned to receive either the professional 
development intervention or business-as-usual professional development. Data collection 
includes measures of teacher knowledge and practice, and student achievement obtained 
from administrative records. 


Estimated or Actual Completion Date: A report on implementation and impacts was released 
in September 2016. 


Key Findings: 


e The professional development had a positive impact on teacher knowledge: Average 
scores on a study-administered math test were 21 percentile points higher for teachers 
who received the study professional development than for those who did not. 


e The professional development had a positive impact on some aspects of instruction: 
Average ratings of teachers’ use and quality of math explanations in the classroom were 
23 percentile points higher for teachers who received the study professional 
development than for those who did not. 


e The professional development did not have a positive impact on student achievement: 
Students of teachers who received the professional development scored, on average, 
2 percentile points lower than control teachers’ students on both a study-administered 
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math test and state assessment. In general, this difference was not statistically 
significant. 


6. Link to Additional Information: http://ies.ed.gov/ncee/pubs/20164010/ 


Other 


Case Studies of Schools Receiving School Improvement Grants: Final Report 


1. 


Study Purpose: School Improvement Grants (SIGs) are authorized by Title |, Section 
1003(g) of the ESEA. The purpose of the grants—awarded based on the Title | funding 
formula to states, which then competitively distribute the funds to districts applying on behalf 
of their eligible schools—is to support the turnaround of the nation’s persistently lowest- 
achieving schools. The first cohort of SIG, which is the focus of this study, received funding 
beginning in the 2010—11 school year. To qualify for the three-year grant in the first cohort, 
schools must (among other requirements) have been willing to implement one of four 
prescribed intervention models: turnaround, restart, closure, or transformation. About 

$546 million was allocated in FY 2009 for SIG with a supplement of $3 billion from the 
Recovery Act. With the possibility of rollover funds, this amounts to a $3.5 billion injection 
into the SIG program during the 2010-11, 2011-12, and 2012-13 school years. This study 
will provide descriptively rich, primarily qualitative information for a small set of schools 
receiving SIG in the first cohort to implement an intervention model beginning in the 
2010-11 school year. 


Key Questions Addressed: 


e What is the background and context of these persistently lowest-achieving schools? 
How do the leadership and staff in these schools define the performance problem, and 
to what do they attribute their problems? 


e What leadership styles do the principals of these persistently lowest-achieving schools 
exhibit? What actions do these schools engage in to try to improve their history of low 
performance? 


e What is the change process in these persistently lowest-achieving schools, particularly in 
terms of school climate and staff capacity? 


e What improvements do school stakeholders perceive during the three-year course of 
SIG, and do these improvements appear to be sustainable? 


Design: This study employs a school-level case study design. A core sample of 25 SIG 
schools was purposively selected from six states to represent a range of geographic 
regions, urbanicities, school sizes, racial/ethnic compositions, socioeconomic statuses, SIG 
intervention models, and SIG funding levels, among other factors. Data collection took place 
over three school years, beginning in spring 2011 and concluding in spring 2013, and 
included interviews with each state’s SIG leaders, a teacher survey, and site visits to the 
case study schools, which included analysis of fiscal records, as well as interviews and 
focus groups with district officials, principals, teachers, parents, union officials, external 
support providers, and students. 


4. Estimated or Actual Completion Date: A final report was released in April 2016. 
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5. Key Findings: 


e Most case study schools (22 of 25) replaced their principal at least once in the year 
before SIG or the first two years of SIG: Two of the four SIG intervention models 
required the principal to be replaced. About half of the new principals were described by 
school staff as an improvement over their predecessor. 


e Twelve of the 25 case study schools replaced at least half of their teachers by the 
second year of SIG: Respondents in more than half of the 12 schools characterized the 
change as positive. All but one of the 25 schools created new nonteaching positions in 
the first two years of SIG, with the most frequent positions being instructional, 
technology, or data coaches and additional school administrators. 


e In 15 of the 25 case study schools, most of the teacher professional development was 
job-embedded: According to teacher survey responses, professional development more 
often focused on math, literacy, or data use than classroom management or improving 
instruction for English learners and special education students. In most schools, 
teachers reported changing their practice after participating in professional learning on 
math, literacy, or data use. 


e Sustaining improvements may be challenging: In more than half of the 12 schools the 
study followed for all three years of SIG, teachers felt their school had changed in 
primarily positive ways. However, just two of these schools show strong prospects for 
sustaining improvement, while six show mixed prospects, and four show weak 
prospects. The schools that had higher organizational capacity by the third year of SIG 
had higher sustainability prospects. 


6. Link to Additional Information: http://ies.ed.gov/ncee/pubs/20164002/ 


Policy and Program Studies Service (PPSS) 


Implementing Accountability and Supports Under ESEA Flexibility 


1. Study Purpose: This study examines the early implementation of the ESEA Flexibility 
initiative at the state and local levels. The study includes reviews of state and local 
documents; interviews with education officials at the state, district, and school level in early 
implementing states; and analysis of baseline student outcome and other data obtained 
through extant data sources. 


2. Key Questions Addressed: 


e What are the primary components of state accountability and support systems under 
ESEA Flexibility? 


e What modes of communication did states and districts use to inform and build local-level 
understanding of state accountability and support systems under ESEA Flexibility? How 
did local-level officials perceive state accountability systems under ESEA Flexibility, 
including the criteria being used to identify low-performing schools? 


e What approaches did states take to identify and intervene in low-performing districts and 
schools? What improvement strategies did principals report implementing? 


e What supports were states and districts providing to low-performing schools? How did 
officials perceive their capacity to implement the ESEA Flexibility provisions for state 
support and accountability systems, and what challenges did they report? 
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Design: The primary data sources for this study included state ESEA Flexibility applications, 
state and district policy documents, and interviews with officials identified by state and 
district leaders as holding primary responsibility for accountability and support systems 
under ESEA Flexibility. 


Actual Completion Date: The report was released in February 2016. 
Key Findings: 


e Indeveloping updated accountability systems, five of the 12 states in the study used a 
combined subgroup as part of their system, and eight of the 12 devised their own goals 
for measuring student achievement. Eight of the 12 created a designation category for 
low-performing schools that did not meet the priority and focus designation criteria. 


e State officials reported providing support to priority and focus schools through regional 
technical assistance organizations, external support providers, and early warning 
systems. Ten of the 13 priority and focus school principals described receiving more 
frequent or more intensive support than in years prior to ESEA Flexibility. 


e State officials described building capacity through cross-agency collaborations, 
developing staff expertise, partnering with districts, implementing organizational 
changes, and consolidating funding streams. 


e Nearly all state officials (11 of 12) reported holding periodic committees, advisory 
councils, and commissions to share ESEA Flexibility update to districts. 


6. Link to Additional Information: http://www2.ed.gov/rschstat/eval/title-i/implementing- 


accountability-esea-flexibility/report.padf 


Evaluation of the Teacher Incentive Fund: Final Report 


1. 


Study Purpose: This study examines program implementation in the first two cohorts of 
grantees for the TIF program (2006 and 2007), which included 33 grantees. All 33 grantees 
implemented performance pay systems for principals and other school administrators; 31 
grantees also included teachers in their performance pay systems. The final report 
complements a previous 2012 report from this study on the early implementation of the 
program, and analyzes award payouts and educator perspectives on a variety of issues 
related to implementation. 


Key Questions Addressed: 


e How did the size and distribution of incentive awards vary across educators and 
grantees? 


e To what extent were incentive payments based on student achievement vs. other factors 
such as supports for improving practice, performance evaluations, and teaching in hard- 
to-staff schools or subjects? 


e How did participating educators perceive the fairness and effectiveness of the 
performance pay system? 


Design: Data collection included surveys of a representative sample of teachers and 
principals in schools served by TIF projects; financial data for awards paid to educators; 
interviews with TIF project directors, teachers, principals, and other stakeholders; and 
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reviews of extant documents. The surveys were conducted in spring 2011, and the incentive 
award payout data are primarily for the 2010-11 school year. 


Actual Completion Date: The report was released in June 2016. 
Key Findings: 


e Across all educators who received an incentive award, the average award was $3,651 
for teachers and $5,508 for administrators. These average awards represented 
approximately 8 percent of average teacher salaries and 6 percent of average principal 
salaries. 


e Within grantees, the average teacher award ranged from $1,170 for the grantee with the 
smallest average award to $8,772 for the grantee with the largest average award 
(2-24 percent of average salaries), and the average administrator award ranged from 
$814 to $10,711 (1-13 percent of average salaries). 


e Awards for student achievement gains comprised the largest share of incentive payouts 
to teachers and administrators; 64 percent of teacher award payouts and 63 percent of 
administrator award payouts were based on student achievement. 


e Smaller proportions of incentive payouts were based on performance evaluations 
(13-14 percent), participation in supports for improving practice (12-14 percent), and 
working in hard-to-staff schools and subjects (6-12 percent). 


e Teachers were less likely than principals to agree that their performance pay system 
was fair (46 percent vs. 64 percent). However, 55 percent of both teachers and 
principals agreed that the possible award size was large enough to motivate them to 
earn it. 


6. Link to Additional Information: 


http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq 


Case Studies of the Implementation and Use of Kindergarten Entry Assessments (KEAs) 


Ts 


Study Purpose: The purpose of the KEAs implementation case studies is to examine the 
processes, accomplishments, challenges, and solutions of four states implementing KEAs 
and to share what they have learned with federal and state policymakers and the field. Of 
particular interest is to identify what is working well in states that are early adopters of KEAs. 
This information is needed to support the technical assistance efforts of the Department and 
to inform KEA efforts across the nation. 


Key Questions Addressed: 


e How did the four case study states develop or adopt KEAs? 


e How did the four states train teachers to administer KEAs and to what extent did 
teachers feel prepared to do so? 


e What were the KEA implementation experiences of the 12 case study districts? 

e To what extent did the states, districts, and schools in the study use KEA results to 
inform policy and practice? 

e What challenges did the case study sites experience with KEAs, and what strategies did 
sites use or Suggest using to address these challenges? 
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. Design: The study consisted of document reviews, telephone interviews with state agency 


respondents and local preschool directors, and in-person interviews with district 
administrators, principals, kindergarten teachers, and other KEA assessors. Data were 
collected in 12 districts and 23 schools in Maryland, Oregon, Pennsylvania, and Washington 
between January and June 2015 and include information from a total of 201 interviews. 


Actual Completion Date: The report was released in August 2016. 
Key Findings: 


e State officials and stakeholders considered multiple criteria when developing or adopting 
KEA measures: reliability and validity, appropriateness for all students, usefulness for 
informing classroom instruction, usefulness for informing early learning policies and 
program improvement, feasibility of administration by teachers, and cost. 


e The four states trained teachers on KEA administration through self-paced webinars, 
in-person presentations, and train-the-trainer models. A majority of the interviewed 
teachers said the training prepared them to administer the KEA to students, though 
many teachers reported that they had difficulty in determining what were appropriate 
accommodations for English learner students and students with disabilities and indicated 
that they needed further assistance. 


e District officials reported working to reduce the burden associated with KEA data 
collection and entry by purchasing new technology, providing staffing assistance to 
teachers with KEA administration, and omitting or delaying other assessments. 


e Although the majority of interviewed teachers reported that they had not yet used formal 
KEA reports to inform their instructional practices, a few teachers said that the 
impressions they gained while administering the KEA helped them to understand their 
students’ strengths and needs and to assign students to instructional groups. 


e District administrators and teachers identified challenges with administering KEAs with 
English learner students and students with disabilities, using KEA results to inform 
instruction, and sharing KEA data with parents; they suggested that state officials could 
address these challenges by providing explicit training on these topics, on-site coaching, 
and tailored reports to help educators use and share the data. 


6. Link to Additional Information: http://www2.ed.gov/rschstat/eval/disadv/kindergarten-entry- 


assessments/report.pdf 


A Study of Practices to Improve Learning in Kindergarten Through Third Grade 


ie 


Study Purpose: This study summarized how (a) aligning preschool through third grade 
education and (b) differentiating instruction for children in kindergarten and first grade may 
build on the positive effects of preschool and help students in kindergarten through third 
grade (K-3) make cognitive, social-emotional, and academic gains. In addition, the study 
conducted case studies of five programs that may build on the positive effects of preschool 
by using policies, programs, and practices for alignment and differentiation. 


2. Key Questions Addressed in the Literature Review: 


e What approaches does the research and theoretical literature suggest for aligning 
preschool through third grade (P—3) education, and what is the quality of the research 
studies? 
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e What are the findings from studies of differentiated instruction on children in kindergarten 
and first grade, and what is the quality of these studies? 


Key Questions Addressed in the Case Studies: 


e What approaches did the five programs use to implement P-3 alignment? 


e In programs that implemented differentiated instruction, what approaches did staff use? 


3. Design: The literature review covered two topics: (1) preschool and K-3 alignment, and 
(2) differentiated instruction in kindergarten and first grade. The case studies examined five 
sites that implemented P-—3 alignment or differentiating instruction in kindergarten and first 
grade. The research team interviewed principals, teachers, evaluators, and funders to 
understand programs’ characteristics, challenges and solutions, and the sustainability of the 
programs. 


4. Actual Completion Date: The literature review was released in August 2016, and the case 
study report was released in December 2016. 


5. Key Findings from the Literature Review: 


e Nearly all qualitative studies and policy and theory articles on P—3 alignment suggest 
aligning standards, curriculum, instruction, assessments, and environments across 
preschool and grades K-3. 


e P-3 alignment could be supported by establishing similar teacher education and training 
requirements and developing longitudinal data systems that integrate preschool and 
K-12 data. 


e Ofthe 17 quantitative studies of differentiated instruction, one RCT of the Individualized 
Student Instruction With Assessment to Instruction intervention demonstrated positive 
results on reading outcomes and had the potential to meet the criteria for strong causal 
evidence. 


e Qualitative studies of differentiated instruction indicate that opportunities for peer 
collaboration and guidance by mentors may be helpful to improve teacher practice 
related to differentiation. 


Key Findings from the Case Studies: 


e All five programs aligned instruction across grades by coordinating standards, curricula, 
instructional practices and professional development. 


e Common elements of these programs included the use of professional learning 
communities, coaches, parent engagement, and play-based or student-initiated learning. 


e All five programs reported using strategies to accommodate students’ different skill 
levels, including modifying assignments, adapting learning materials, providing different 
levels of support, or using small-group instruction. 


6. Links to Additional Information: 


Literature review: http:/Awww2.ed.gov/rschstat/eval/disadv/p-3-alignment-differentiated- 
instruction/report.pdf 


Case studies: http://www2.ed.gov/rschstat/eval/implementing-early-strategies/report.paf 
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Study of Emerging Teacher Evaluation Systems 


Ts 


Study Purpose: This study will provide descriptive information on the design and early 
implementation of teacher evaluation systems in eight school districts. The findings are 
intended to help other districts and states learn from the experiences of eight districts 
featured in the study, and apply the design and implementation lessons to their own work as 
it relates to teacher evaluation and support. 


Key Questions Addressed: 


e What key priorities and measures informed the design of the new teacher evaluation 
systems? 


e What steps did the districts take prior to full implementation to test the system and 
prepare teachers and staff to implement it? 


e How did the districts structure and conduct the classroom observation component of 
their teacher evaluation systems? 


e How did the districts analyze student performance and other data to evaluate teacher 
performance? 


e How did the districts use, or plan to use, teacher evaluation results to make personnel 
decisions? To what extent were professional development and career advancement 
decisions tied to evaluation results? 


e What administrative structures did districts use to support their new teacher evaluation 
systems? 


e What are the perceived early effects of the teacher evaluation systems on the 
professional practices of teachers, principals and district administrators? 


Design: This descriptive study relied on interviews with key district administrators, principals, 
teachers, and representatives of community stakeholder groups, from eight districts, who 
were involved in the development and early implementation of the respective districts’ 
teacher evaluation system. Given the limited sample, the findings cannot be generalized to 
other districts. 


Actual Completion Date: The final report was released in November 2016. 

Key Findings: 

e Teachers and central office staff generally agreed that the foremost goal of the teacher 
evaluation system was to improve instruction. 


e Teacher and principal input during the design and/or pilot test phase strongly influenced 
decisions regarding system modifications in six districts, according to district 
administrators. 


e Classroom observations varied in frequency, duration, and degree of formality in all eight 
districts. In addition, principals reported challenges in finding time to conduct teacher 
observations. 


e Six districts used multiple approaches for measuring teacher impact on student 
performance, including individual and/or school-level value-added models. 
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e Districts used teacher evaluation results for a range of purposes, including targeted 
professional development and support, career ladders and performance pay, and in 
some instances, redeployment or release of teachers identified as ineffective. 


e The majority of districts created relatively simple, streamlined structures to administer 
their teacher evaluation systems. 


e Teachers reported that they believed that the classroom observations and feedback 
helped them become better teachers. 


6. Link to Additional Information: 
http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq 


State Efforts to Promote Equitable Access to Effective Teachers 


1. 


Study Purpose: This report provides a broad overview of state efforts, as of the 2011-12 
school year, to monitor equitable access to qualified and effective teachers among schools; 
develop and adopt multiple measures of teacher performance to rate teachers among at 
least three performance levels; and implement targeted strategies for promoting equitable 
access to qualified and effective teachers in schools serving high proportions of poor and/or 
minority students. The report examines the use of measures of teacher qualifications and 
teacher performance in the implementation of these strategies. 


Key Questions Addressed: 


e What measures did states use to monitor equitable access to qualified and effective 
teachers among schools? 


e To what extent were states developing or using multiple measures of teacher 
performance to rate teachers among at least three performance levels? 


e What strategies did states use to promote equitable access to qualified and effective 
teachers in schools serving high proportions of poor and/or minority students? 


Design: This report is based on telephone interviews with officials in SEAs in all states, the 
District of Columbia, and Puerto Rico. Study staff also reviewed extant sources including 
equity plans, Consolidated State Performance Reports, reports from federal monitoring 
visits, and other information on state websites. The interviews were conducted between 
August 2011 and January 2012, and the study reflects state efforts that were underway at 
that time, and predates implementation of the fall 2011 ESEA Flexibility initiative as well as 
the recent state equity plans submitted under the Excellent Educators for All initiative. 


Actual Completion Date: The final report was released in January 2017. 

Key Findings: 

e In 2011-12, states most commonly monitored equitable access to qualified and effective 
teachers among schools using measures of teacher qualifications. 


e Four states reported using measures of teacher performance—student achievement 
growth and/or measures of teacher practice—to monitor equitable access to qualified 
and effective teachers among schools, and nine states used teacher performance 
measures to monitor the quality of the teacher workforce overall. 
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e In two of the four states that reported using teacher performance measures to monitor 
equitable access among schools, officials reported seeing larger inequities than were 
previously detected using measures of teacher qualifications alone. 


e Six states reported that they had adopted multiple measures of teacher performance and 
were using them to rate teachers among at least three performance levels in 2011-12, 
and 38 states indicated that there were in the process of developing such measures. 


e Offering monetary incentives was the most common strategy that states reported using 
in disadvantaged schools to promote equitable access to qualified and effective teachers 
among schools (24 states). 


e Other state-reported strategies that were directed specifically at disadvantaged schools 
for promoting equitable access were specialized professional development (14 states) 
and teacher recruitment and preparation programs (14 states). 


6. Link to Additional Information: 
http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq 


Feasibility Study on Improving the Quality of School-Level Expenditure Data 


Ts 


Study Purpose: This study explores the feasibility of improving the collection of school-level 
expenditure data by examining the nature and quality of school-level fiscal data collection in 
five states and four school districts that had developed their own systems for collecting and 
reporting school-level expenditures: Florida, Hawaii, Ohio, Rhode Island, Texas, Baltimore 
City, Hillsborough County, Houston, and Los Angeles 


Key Questions Addressed: 


e Instates and districts that have developed systems to report expenditures at the school 
level, what types of personnel and nonpersonnel expenditures are included in the 
school-level data? 


e To what extent do the sites track actual expenditures to individual schools versus 
allocating or prorating expenditures to schools using formulas? 


e How consistent are school-level expenditure data obtained from these systems with 
similar data from other sources? How do the funding amounts attributed to individual 
schools based on formula allocations compare to those based on tracking actual 
expenditures? 


e What lessons can other states and districts learn from these sites if they wish to 
implement systems for reporting accurate and reliable data on school-level 
expenditures? 


Design: The study included (1) surveys and interviews of officials to understand the process 
of collecting school-level expenditure data and (2) collection and analysis of school-level 
spending data to examine data quality issues. The study examined three aspects of data 
quality: the comprehensiveness of school-level spending data, consistency with other data 
sources, and the relative accuracy of allocating expenditures to schools by formula (rather 
than tracking actual expenditures for each school). 


4. Actual Completion Date: The final report was released in January 2017. 
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5. Key Findings: 


Study sites reported that they attributed most categories of spending to the school level, 
including salaries for teachers, administrators, and other support staff as well as 
nonpersonnel items, such as textbooks, instructional materials, furniture and equipment, 
and computers and software. 


The school districts and states in this study attributed an average of three-quarters of 
operational expenditures to individual schools, demonstrating that it is feasible to link a 
significant share of spending to the school level. 


Most of the expenditures that the study sites attributed to schools were directly tracked 
to schools (85 percent) rather than simply being allocated by formula (8 percent). 


Comparisons between the site-reported school-level expenditures and other data 
sources showed a relatively high degree of consistency for salary expenditures, but 
nonpersonnel expenditure data were much less consistent. 


Allocating expenditures to schools by formula (e.g., based on total salaries or staff) 
appeared relatively accurate for health benefits and less accurate for pension benefits, 
pupil support staff, and instructional support staff. 


Instituting a system for collecting school-level expenditure data typically required new 
hardware and software (eight sites), changes to charts of accounts (six sites), and staff 
training (eight sites). 


Advice that interviewees offered for others aiming to implement school-level expenditure 
data systems was to get stakeholders involved, communicate clearly and frequently, and 
think long-term about future data needs. 


6. Link to Additional Information: 
https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#title 


Study of Experiences and Needs of Rural Education Achievement Program (REAP) 
Grantees and Subgrantees 


1. Study Purpose: This descriptive study examined how grantees and subgrantees use REAP 
funds provided through the Small, Rural School Achievement (SRSA) and Rural and Low- 
Income Schools (RLIS) programs—on their own or in combination with other federal funds— 
as well as to explore technical assistance needs related to REAP. 


2. Key Questions Addressed: 


What role do states play in supporting the Department’s SRSA and RLIS eligibility and 
award determination process? 


How do districts use their SRSA or RLIS funds? 
To what extent do SRSA-eligible districts use REAP Flex and for what purposes? 


What recommendations do states and districts have for improving the operation of the 
SRSA and RLIS programs? 


3. Design: The study consisted of: 1) a survey of a sample of approximately 1,000 SRSA 
grantees and RLIS subgrantee districts; 2) telephone interviews with a sample of 30 SRSA 
grantees and RLIS subgrantees; and 3) telephone interviews with REAP coordinators in all 
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states receiving REAP funds. Data collection began in winter 2015 and was completed by 
spring 2015. At the state level, the study included interviews with all state REAP 
coordinators about state goals and priorities, the planning process for use of RLIS funds, the 
eligibility process for districts, management and distribution of SRSA and RLIS funds, and 
recommendations for the program. At the school district level, the study included an online 
survey of a nationally representative sample of REAP coordinators about the REAP 
eligibility determination process, administration of REAP, and challenges and technical 
assistance needs related to REAP, as well as telephone interviews with a subsample of 

30 districts. The interviews included questions about program administration, technical 
assistance needs, and recommendations for changing the REAP program to better meet the 
needs of rural districts. 


4. Actual Completion Date: The report was completed in December 2016. 
5. Key Findings: 


e States supported the Department in determining REAP eligibility by providing district- 
level data and reviewing the accuracy of Department-provided data. 


e All 43 states with RLIS-eligible districts chose to make subgrants to districts on the basis 
of a funding formula rather than on a competitive basis, and 28 of these states based the 
subgrant amount entirely on average daily attendance (ADA). 


e Districts most frequently used SRSA and RLIS funds to improve or expand access to 
technology (71 percent of SRSA districts and 71 percent of RLIS districts) and to provide 
educator professional development (45 percent of SRSA districts and 58 percent of RLIS 
districts). 


e Forty-six percent of SRSA district coordinators reported exercising REAP-Flex; of these, 
82 percent reported that they used funds eligible for REAP-Flex to maintain a stable 
level of funding for ongoing activities. 


e The majority of both district and state REAP coordinators were highly satisfied with 
REAP as a whole. However, they provided recommendations for improvement to REAP 
in three categories: (1) improved timelines for eligibility and award determination, 

(2) more information on allowable uses of funds and REAP-Flex, and (3) revised 
eligibility criteria. 


6. Link to Additional Information: 
https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#rural 
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Summary of Performance Evaluations Expected During FY 2017 and 
FY 2013" 
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Pathways to Career or College 
Enhancing Advising to Improve College Fit in Upward Bound 


1. Study Purpose: Growing concern over college enrollment and completion rates has 
heightened interest in cost-effective strategies to improve the outcomes of low-income 
students, including those in high-profile college access programs. One aspect of this 
concern is that many low-income high school students fail to enroll in colleges that are well 
aligned with their qualifications, talents, and needs. This report provides the early impacts of 
a low-cost enhancement to current college advising approaches in Upward Bound that is 
designed to improve college fit and persistence. The approach includes professional 
development for Upward Bound advisors and user-friendly packets of information for 
students that demonstrate the value of considering a range of institutional indicators in their 
college application, search, and acceptance process. 


2. Key Questions Addressed: 


e Can an enhanced college advising approach improve upon what Upward Bound 
grantees are already doing—i.e., does it have positive effects on the number of colleges 
to which Upward Bound participants apply, the quality/selectivity of the colleges in which 
they enroll, and their persistence? 


e In what types of grantees is this approach most effective and with what types of 
students? 


3. Design: About 200 Upward Bound grantees that volunteered were randomly assigned in 
spring 2015 so that half received the professional development and packets to begin using 
with their rising 2015-16 seniors (the treatment group) and half did not receive the 
enhancements for use with rising 2015-16 seniors (the control group); however, the control 
group did receive training and packets a year later to use with rising 2016-17 seniors. A 
survey administered in spring 2016 to seniors in both sets of projects collected information 
about their college planning, including the number of applications. The study will also 
examine those seniors’ later enrollment and persistence in college using administrative 
records. 


4. Estimated or Actual Completion Date: The first report on early impacts (number of college 
applications, FAFSA completion by March 15) is expected in fall 2017. A second report 
examining impacts on college enrollment and fit is expected in early 2018. 


5. Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/pathways upward.asp 


“This section was last updated in April 2017. Key findings are included for reports that were released during the first half of FY 2017 
(October 2016 to March 2017). All other reports that are expected but not yet released in the second half of FY 2017 or in FY 2018 
have estimated completion dates only. 
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Is Access to Data Enough? The Effects of Providing Information on Student FAFSA 
Completion Directly to School Districts 


Ts 


4. 
5. 


Study Purpose: Completing a Free Application for Federal Student Aid (FAFSA) is a key but 
challenging task for many students applying to college, particularly for low-income students. 
The Department has responded to concerns about the complexity of the FAFSA, including 
through the FAFSA Completion Project which started with a 20-district pilot in 2010 and 
expanded in 2012 to almost 100 more. One component of the project was consistent urging 
by federal officials and some technical assistance for participating districts to help them 
conduct FAFSA promotion and outreach activities districtwide. But the Department also 
provided districts with ongoing and direct access to federal data on which seniors had not 
submitted a FAFSA so that schools could provide targeted support to individual students. 
The evaluation tested whether that data access had an impact on FAFSA completion rates 
above and beyond the other FAFSA activities the districts might have implemented. The 
study was conducted by IES staff and designed to be low-burden, and low-cost, depending 
only on the Department’s data for measuring effectiveness. 


Key Question Addressed: 
e Does making the student-level data available improve FAFSA completion rates? 


Design: Just over 60 districts that had more than one high school participated in the 
evaluation, for a total of 567 schools. IES randomly assigned the schools within each district 
to either an “early start” (treatment) group, which would have access to the FAFSA 
completion status for 2013 seniors, or a “later start” (control) group, which would have 
access for 2014 seniors. Because many districts had relatively small numbers of schools, 
pairs of schools were matched prior to random assignment to help ensure that important 
characteristics of schools were balanced between the treatment and control groups. At the 
beginning of the evaluation, districts provided the lists of all seniors in each school and 
these were matched against the Department’s federal student aid database so that FAFSA 
completion rates in March 2013 and September 2013 could be compared between the two 
groups of schools. 


Estimated or Actual Completion Date: The report is expected in fall 2017. 


Link to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/pathways fafsa.asp 


The Effectiveness of Text Messaging to Support College Transition of GEAR Up Students 


a 


Study Purpose: GEAR Up is one of the U.S. Department of Education’s college access 
programs, funding states or local partnerships of districts and postsecondary institutions to 
serve students in high need schools beginning in 7th grade. The 2008 amendments to the 
Higher Education Act (HEA) allowed GEAR Up grantees to serve participating students 
beyond high school graduation and into a first year of college, when they might be 
dispersing to a variety of college campuses. This demonstration evaluates a low-cost way to 
provide these services, building on emerging evidence that customized text messages can 
help students overcome logistical and behavioral challenges that might otherwise derail their 
college matriculation and persistence into sophomore year. The messages include 
reminders and information relating to college registration, course selection, financial aid 
award and renewal, meeting with college advisors and faculty, and tuition payments, and 
provide a way to access real-time support from GEAR Up counselors. 
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Key Questions Addressed: 


e Do the text-based reminder strategies improve GEAR Up students’ rates of college 
enrollment and persistence? 


e For which types of students is the strategy more or less beneficial? 


Design: The first grantees eligible to use funds to support a year of post-high school 
services and who have seniors entering that phase have been targeted for the 
demonstration. Across about 80 schools, approximately 6,000 seniors in school years 
2015-16 or 2016-17 will be randomly assigned to receive either the college transition 
services grantees originally proposed in their applications or those regular transition 
services plus the customized reminders and support through text messages. Reminders and 
support will begin at the end of students’ high school senior year and continue into the 
spring of their expected first year of college. The study team will administer a survey before 
the reminder messaging begins, to collect information on students’ experiences with college 
advising and their intended college (so that the messages can be tailored to individual 
schools’ deadlines and requirements). College enrollment and persistence, as well as 
FAFSA renewal, will be measured using administrative records. 


Estimated or Actual Completion Date: The first report examining impacts on initial college 
enrollment is expected in fall 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/pathways gearup.asp 


Upward Bound at 50: Reporting on Implementation Practices Today 


1. 


4. 


Study Purpose: Upward Bound is the oldest and largest of the federal college access 
programs targeted to low-income students and those who would represent the first- 
generation of college completers in their families. First established in 1965, the program 
currently serves more than 60,000 high school students at a cost of about $4,300 per youth 
with a wide array of academic and college transition support services. While much about the 
structure of Upward Bound and the services to be offered are prescribed in legislation, little 
is currently known about the intensity, duration, and mix of services provided by projects or 
about how they are delivered. Because of the importance of its mission, and the 
comprehensiveness and costs of its services, Upward Bound has long been of interest to 
policymakers. This report describes the approaches to providing program services as 
reported by Upward Bound project directors. 


Key Questions Addressed: 


e Within the core service areas of the program, where do projects focus their efforts? 
e How are services delivered to Upward Bound participants? 


e In what ways does the focus or delivery of services vary across different Upward Bound 
projects? 


Design: This descriptive report relies primarily on findings from a summer 2013 survey of all 
regular Upward Bound project directors. 


Estimated or Actual Completion Date: The report was released in November 2016. 
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5. Key Findings: 


e In four of the seven core service areas—coursework, tutoring, college exposure, and 
college application assistance—there was a dominant approach (used by at least 
50 percent of projects) to how projects focused their activities. There was no dominant 
approach to how projects focused their efforts when it came to academic advising, 
ACT/SAT prep, and financial aid prep services. 


e When, where, and how services were delivered differed across service areas. There was 
no dominant approach to when projects offered services except for tutoring, which was 
typically available after school. The dominant location for services (where) was at the 
projects’ host institution for coursework, college entrance exam prep, and college and 
financial aid application assistance, but other services were more likely to be provided at 
students’ high schools. Finally, tutoring and college entrance exam preparation services 
were most commonly delivered (how) in groups, while academic advising, college 
application assistance, and financial aid assistance were typically provided one-on-one. 


e Variation in the focus and delivery of services appears related to the urbanicity and type 
of institution (four-year, two-year, and nonhigher education) that hosts the project but not 
to other project characteristics examined. There were few substantive differences (at 
least 10 percentage points) in the percentage of projects reporting each potential 
approach by project size (number of students served), per-student funding, and whether 
the host institution was a Minority-Serving Institution. 


6. Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174005/ 


School Choice 


Evaluation of the DC Opportunity Scholarship Program: Impacts After One and Two 
Years 


1. 


Study Purpose: The April 2011 SOAR Act provided for a five-year continuation of a school 
choice program for low-income residents of Washington, DC. The program, still titled the 
Opportunity Scholarship Program (OSP), now provides annual scholarships of $8,000 (for 
grades K-8) or $12,000 (for grades 9-12) to enable low-income students to attend private 
schools in DC in lieu of the public schools already available to them. The new law also 
mandated another independent, rigorous evaluation of the effectiveness of the program. 


Key Questions Addressed: 


e What is the impact of the OSP on student academic achievement and other measures of 
student success, overall and for subgroups of students identified in the statute as high 
priority? 


e What effect does the program have on student and parent perceptions of school safety 
and satisfaction, and on parents’ involvement in education of their children? 


e Does the program change students’ instructional environments and opportunities? 


Design: The evaluation will primarily compare outcomes of approximately 1,800 student 
applicants randomly assigned by lottery to either receive a scholarship or not receive a 
scholarship. Lotteries of program applicants were conducted in spring 2012 (cohort 1), 
spring 2013 (cohort 2), and spring 2014 (cohort 3). Data will be collected for three follow-up 
years for each of the cohorts and, for students in both the scholarship and nonscholarship 
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groups. The contractor will administer academic assessments, and conduct student, parent, 
and principal surveys each spring (spring 2013-16). Prior to the first impact report, 
descriptive reports based on application forms parents filled out when they applied to the 
OSP, principal surveys, and extant data were released. 


Estimated or Actual Completion Date: The first impact report is expected in spring 2017. The 
second impact report is expected by early 2018. 


5. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/choice_soar.asp 


Parent Information and School Choice Evaluation 


1s 


5: 


Study Purpose: School choice is among the most visible K-12 education policy trends to 
emerge over the last two decades. For school choice to be effective, it seems critical that 
parents are able to navigate school choice systems and process large amounts of complex 
information about schools and application procedures to make informed choices. However, 
few studies have rigorously examined school choice information strategies or attempted to 
identify effective methods of information presentation. This evaluation is designed to 
address this significant gap in the literature. 


Key Questions Addressed: 


e What is the optimal way to present school choice information? (E.g., which presentation 
formats make school performance/achievement data easiest to understand?) 


e What is the right amount of information to provide to parents whom might be making a 
school choice? (E.g., is there a limit to the number of choice options and amount of 
information that parents can reasonably consider?) 


e Howis school choice information best organized? (E.g., does the order in which 
information is presented shape how it is understood and used? Is there a way to make 
the information parents most often report caring about salient?) 


Design: The evaluation findings will be based on low cost quick turn-around “lab” based 
experiments. These studies will take advantage of opportunistic samples of low income 
parents in order to test a large number of potentially promising information strategies and to 
obtain findings within a relatively short period of time. 


Estimated or Actual Completion Date: A short user-friendly guide, based on the evaluation’s 
findings, that will help school districts and other providers of school choice information 
design or refine their own parent information materials is expected in October 2017. 


Link to Additional Information: http://ies.ed.qov/ncee/projects/evaluation/choice_parent.asp 


Students with Disabilities 


Impact Evaluation of Training in Multi-Tiered Systems of Support for Behavior (MTSS-B) 


1; 


Study Purpose: Training school staff in supporting student behavior is becoming 
increasingly attractive to districts and schools as a vehicle for school improvement. 
Implementation of MTSS-B is an approach to improving school and classroom climate as 
well as student outcomes. MTSS-B is a multi-tiered, systematic framework for teaching and 
reinforcing behavior for all students as well as for providing additional support to those who 
need it. Over a third of U.S. districts report implementing MTSS-B at the elementary school 
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level. Recent studies have shown the promise of MTSS-B, and a large-scale study of the 
effectiveness of MTSS-B is needed. 


2. Key Questions Addressed: 


e What MTSS-B training and support activities were conducted? What MTSS-B activities 
occurred in the schools receiving MTSS-B Training? How do these MTSS-B activities 
differ from those in schools that do not receive the training? 


e What is the impact on school staff practices, school climate and student outcomes of 
providing training in the MTSS-B framework plus universal (Tier |) positive behavior 
supports and a targeted (Tier II) intervention? 


e What are the impacts for relevant subgroups (e.g., at-risk students)? 


3. Design: This is a randomized trial of the impact of training in MTSS-B on school climate, 
school staff practice, and student outcomes. The contractor, with assistance and input from 
the Department and in consultation with a panel of experts, selected a MTSS-B training 
provider, the Center for Social Behavior Support (CSBS), which is a collaboration between 
the Illinois-Midwest PBIS Network at the School Association for Special Education in 
DuPage, Illinois (SASED) and the PBIS Regional Training and Technical Assistance Center 
at Sheppard Pratt, in Maryland. Approximately 90 elementary schools will be randomly 
assigned to either training in MTSS-B including universal supports (Tier |) plus targeted 
interventions for at-risk students (Tier II) or a business-as-usual control group. Treatment 
schools will receive training in MTSS-B prior to and across two school-years, 2015-2016 
(Tier l) and 2016-2017 (Tiers | and II), and implement MTSS-B across these two years. 
Data collection will include staff surveys, teacher ratings of student behavior, classroom 
observations, site visits, and student records data. 


4. Estimated or Actual Completion Date: The report is expected in 2018. 


5. Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities MTSSB.asp 


National Evaluation of the IDEA Technical Assistance and Dissemination Program: Final 
Report 


1. Study Purpose: IDEA, which was most recently reauthorized in 2004, provides funds to 
assist states and local educational agencies in making available a free, appropriate public 
education for children with disabilities. Funded at $12.6 billion in FY 2010, IDEA supports 
early intervention services for infants and toddlers, special education and related services 
for children ages 3 through 21, and early intervening services for students not in special 
education but in need of academic or behavioral support. IES is conducting studies under 
Section 664 of IDEA to assess the implementation and effectiveness of key programs and 
services supported under the law. 


As specified in IDEA Part D, the Technical Assistance and Dissemination (TA&D) program is 
to provide technical assistance, support model demonstration projects, disseminate useful 
information, and implement activities that are supported by scientifically based research to 
meet the needs of children with disabilities. The National Evaluation of the IDEA TA&D 
program is designed to describe the products and services provided by the TA&D program 
grantees, state and local needs for technical assistance, and the role that the TA&D 
program plays in meeting these needs and supporting implementation of IDEA. The State 
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Deaf-Blind Projects are part of the TA&D Program and are the focus of the evaluation’s final 
report. 


Key Questions Addressed: 


e What technical assistance and dissemination activities do State Deaf-Blind Projects 
provide and how does this vary across the states? 


e How do State Deaf-Blind Projects collaborate with other organizations in their state, with 
other technical assistance providers, and across the network of State Deaf-Blind 
Projects? 


e What are the needs for technical assistance among direct service providers who work 
with children and youth with deaf-blindness? 


e How satisfied are direct service providers with services received from the State Deaf- 
Blind Projects? 


Design: Data collection for the interim report included administering surveys to the national 
and regional level TA&D program grantees, all state IDEA Part B and Part C administrators, 
and a sample of state-level special education program staff. Data were collected between 
November 2012 and March 2013. State-level administrators and staff reported on their 
receipt of technical assistance from TA&D program grantees during the 2010-11 school 
year. Grantees reported on their goals and activities from the beginning of their current grant 
through the interview date. The funding period for the centers included in the interim report 
varied, with the earliest end date in 2012 and the latest in 2014. Data collection for the final 
report included administering surveys to the 48 state level deaf-blind projects awarded 
grants in 2013 and to a sample of local level special and general education providers 
currently working with deaf-blind children. 


Estimated or Actual Completion Date: The final report is expected in summer 2017. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities idea2004.asp 


Preparing for Life After High School 


T 


Study Purpose: Phase | of the National Longitudinal Transition Study 2012 (NLTS 2012), 
also referred to as the Study of Transition Outcomes for Youth with Disabilities Phase I, is 
the third in a series examining the characteristics and school experiences of a nationally 
representative sample of youth with disabilities. NLTS 2012 focuses on youth ages 13 to 21 
(in December 2011), but also includes a small sample of students without disabilities to 
enable direct comparisons of students with and without individualized education programs 
(IEPs). It is part of the congressionally mandated National Assessment of the IDEA and is 
supported with funds authorized under Section 664 of IDEA. 


2. Key Questions Addressed: 


e What are the personal, family, and school characteristics of youth with disabilities in 
public schools across the country? 


e What regular education, special education, transition planning, and other relevant 
services and accommodations do youth with disabilities receive? 
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How do the services and accommodations differ from those of youth not served under 
IDEA, including those identified for services under Section 504 of the Rehabilitation Act? 


How do the services and accommodations for youth with disabilities vary with the 
characteristics of youth? 


How much have the services and accommodations of youth with disabilities changed 
over time? 


3. Design: Phase | of the study collected survey data (spring 2012-summer 2013) ona 
nationwide sample of 12,000 youth in school, of which 10,000 are students with IEPs across 
the federal disability categories. Surveys of both youth and their parents/guardians were 
administered. 


4. Estimated or Actual Completion Date: A report assessing existing research on the 
effectiveness of transition strategies for youth with disabilities was released August 2013. 
Two volumes of a report describing the survey results were released in March 2017. The 
third and final volume of this report is expected later in 2017. 


5. Key Findings from the first two volumes of the March 2017 report include: 


Youth with an IEP, particularly those with intellectual disability and emotional 
disturbance, are more likely than their peers to be socioeconomically disadvantaged. 
Youth with an IEP are 12 percentage points more likely to live in low-income households 
and are less likely to have parents who are employed or have a college education. 
Among disability groups, youth with intellectual disability and youth with emotional 
disturbance are more socioeconomically disadvantaged and more likely to attend a 
lower-performing school than youth with an IEP overall. In contrast, youth with autism 
and youth with speech or language impairments are less socioeconomically 
disadvantaged and less likely to attend a lower-performing school than youth with an IEP 
overall. 


The vast majority of youth with and without an IEP feel positive about school, but those 
with an IEP experience bullying and are suspended at higher rates. Like their peers, 
more than 80 percent of youth in special education report that they are happy with 
school and with school staff. However, not only do youth with an IEP more commonly 
experience some types of bullying (e.g., being teased or called names) but, according to 
parent reports, they are more than twice as likely to be suspended or expelled from 
school. Among the disability groups, youth with emotional disturbance are most likely to 
report being teased and are suspended, expelled, and arrested at more than twice the 
rates of youth with an IEP on average. 


Youth with an IEP are more likely than other youth to struggle academically, yet less 
likely to receive some forms of school-based support. Half of all youth with an IEP report 
they have trouble with their classes, about 15 percentage points more than reported by 
their peers. However, they are less likely to report receiving school-based academic help 
before or after regular hours, although their parents more commonly help with homework 
and attend a parent-teacher conference. Among youth with an IEP, those with autism, 
intellectual disability, and multiple disabilities are least likely to receive school-provided 
instruction outside of school hours though most likely to receive modified tests and 
assignments. 


Youth with an IEP lag their peers in planning and taking steps to obtain postsecondary 
education and jobs. Substantially fewer youth with an IEP expect to enroll in 
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postsecondary education or training, compared to youth without an IEP. Reflecting these 
gaps, youth in special education are almost half as likely as their peers to report taking 
college entrance and placement tests. Forty percent report having recent paid work 
experience while in high school, compared with 50 percent of youth without an IEP. 
Among youth with an IEP, the three groups least likely to receive academic supports 
before or after school—youth with autism, intellectual disability, and multiple 
disabilities—are also least likely to take these steps to prepare for college and 
employment. 


e Youth with autism, deaf-blindness, intellectual disability, multiple disabilities, and 
orthopedic impairments are most at-risk for not transitioning successfully beyond high 
school. Youth in these groups are less likely than all youth with an IEP to have key 
characteristics and experiences linked to success after high school, such as performing 
typical daily living tasks, engaging with friends and in school activities, or preparing for 
college, careers, and independent living. 


6. Link to Additional Information: 


https://ies.ed.gov/ncee/projects/evaluation/disabilities nlts2012.asp 


Preschool Special Education Programs and Practices Supporting Children with 
Disabilities 


1. 


Study Purpose: Most recently authorized in 2004, IDEA provides funding to states to support 
special education and related services for children and youth with disabilities, including 
young children ages 3-5 years old. Currently there is limited information available on the 
curricula and interventions being used across states to support young children with 
disabilities. Phase | of the Evaluation of Preschool Special Education Practices is being 
conducted to address the primary objective of assessing the feasibility of conducting a large- 
scale impact study of curricula or interventions promoting the literacy, language, and/or 
social-emotional skills of preschool-age children with disabilities. A secondary objective of 
the Phase | study is to provide nationally representative descriptive information about 
preschool special education programs and the specific curricula or interventions being 
delivered to preschool children with disabilities. 


Key Questions Addressed: 


e Which curricula and interventions are used nationally for preschool children with 
disabilities to promote learning of language, literacy, and social emotional skills? 


e How are states and school districts structuring programs to serve children ages 3 
through 5 with disabilities? 


e What staff resources are available to support the instruction of children ages 3 through 5 
with disabilities? 


Design: The study collected survey data from state and district grantees of IDEA funds to 
obtain nationally representative information on the programs, services, curricula and 
interventions available to children ages 3-5 identified for special education services. The 
survey sample included a nationally representative sample of 1,200 school district preschool 
special education coordinators and state Section 619 coordinators in all 50 states and DC. 
Additionally, an evidence review will be conducted to identify preschool curricula and 
interventions focused on improving literacy, language, and social emotional skills of 
preschool-aged children with disabilities for the design of impact study options. If an impact 
study is feasible to conduct, preparations for the impact study will occur under Phase I. 
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Estimated or Actual Completion Date: A descriptive report on the survey findings is 
expected in 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities preschool.asp 


Study of Early Intervention and Special Education Services and Personnel 


Is 


4. 
D: 


Study Purpose: IDEA was most recently authorized in 2004 to promote a free appropriate 
public education for children with disabilities. Funded at $12.5 billion in FY 2015, IDEA 
supports early intervention services for infants and toddlers (IDEA Part C), special education 
and related services for children ages 3 through 21 (IDEA Part B), and early intervening 
services for students not in special education but in need of academic or behavioral support. 
Under Section 664 of IDEA 2004, IES is conducting studies to assess the implementation 
and effectiveness of key programs and services supported under the law. This study is 
supporting the analysis of extant data to examine early intervention and special education 
service delivery and the personnel providing services. 


Key Questions Addressed: 


e How has the delivery of early intervention services changed over time, and how do these 
services differ for subgroups defined by age and state? 


e How have special education services changed over time, and how do these services 
differ for subgroups defined by age, disability category, and state? 


e How has the distribution of personnel providing special education services changed over 
time? 


Design: This study includes new descriptive analysis of extant data available in public use or 
restricted formats. Among the data sources that are being used are cross-sectional Section 
618 data submitted by states to the Department and the NCES Schools and Staffing Survey. 


Estimated or Actual Completion Date: The report is expected in summer 2017. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities persserv.asp 


Teacher Effectiveness 


Does Content-Focused Teacher Professional Development Work? Findings from Three 
Institute of Education Sciences Studies 


Ts 


Study Purpose: Federal and local governments continue to invest billions of dollars each 
year in professional development for teachers. Until recently, there has been little rigorous 
evidence to inform the design and delivery of these professional development programs. 
Nevertheless, there has been growing consensus that deepening teachers’ content 
knowledge is an essential component of effective professional development in both reading 
and mathematics. Over the past decade, IES conducted three large-scale random 
assignment studies of teacher professional development in different grades in reading and 
math. These studies, which are the focus of this brief, reveal a common pattern of findings 
on the impact of intensive, content-focused professional development on teaching and 
learning. The findings also highlight unresolved issues that future research might explore to 
advance our understanding and inform professional development policy and practice. 
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2. Key Questions Addressed: This brief synthesizes findings from three large-scale random 
assignment studies of professional development that were conducted by the National Center 
for Education Evaluation and Regional Assistance in IES. Each study examined the impact 
of teacher professional development on teacher knowledge, practice, and student 
achievement. 


3. Design: Although the professional development programs in each study were different, they 
all emphasized building teachers’ content knowledge or knowledge about content-specific 
pedagogy. The programs combined summer institutes with periodic teacher meetings and 
coaching during the school year. These programs were compared to the substantially less 
intensive professional development that teachers typically received in study districts. The 
three studies included 270 second-grade reading teachers, 165 fourth-grade math teachers, 
and 195 seventh-grade math teachers. 


4. Estimated or Actual Completion Date: The brief was released in November 2016. 
5. Key Findings: 


e The professional development improved teachers’ knowledge and some aspects of their 
practice; 


e Improving teachers’ knowledge and practice did not lead to positive impacts on student 
achievement; 


e Most of the measured aspects of teachers’ knowledge and practice were not correlated 
with student achievement; and 


e The consistent pattern of findings suggests that future studies might seek to better 
understand on what aspects of teacher knowledge and practice professional 
development should focus, and how professional development can achieve a larger 
impact on knowledge and practice that also impacts student achievement. 


6. Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174010/ 
Impact Evaluation of Professional Development for Teachers on Data-Driven Instruction 


1. Study Purpose: Title Il, Part A of ESEA funds a broad array of activities to improve teacher 
quality, including professional development. One possible focus of professional 
development is how to use ongoing data from student performance and assessments. 
These data provide information about students’ academic needs and are already available 
to principals and teachers but may not be used effectively. This study evaluates the 
effectiveness of professional development for teachers to use such data to guide instruction. 
Existing evidence on data-driven instruction strategies is limited; however, it suggests that 
professional development that helps teachers focus on individual students’ academic needs 
is promising. 


2. Key Questions Addressed: 


e What are the effects on student achievement of providing professional development to 
principals and teachers that focuses on using individual student academic information 
already available to school staff to guide instruction? 
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e What are the effects of providing this professional development on teachers’ and 
principals’ use of individual student academic information to inform instructional support, 
planning, and practice? 


3. Design: The study is a randomized controlled trial with a sample of 104 schools in 


12 districts. In each district, schools were randomly assigned to treatment (professional 
development) or to control (business as usual) status. Data collection will consist of a 
teacher survey, a principal survey, and student administrative records, including student 
state standardized achievement test scores. 


4. Estimated or Actual Completion Date: The report is expected in summer 2018. 


5. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_datadriven.asp 


Impact Evaluation of Teacher and Leader Evaluation Systems 


Ts 


Study Purpose: The study is designed to examine the implementation of a package of 
performance evaluation system components and the impact of their use for formative 
purposes. These are components that states and districts might elect to include in their 
evaluation systems, with support from Title Il Part A funds under the ESSA. The 
components include measures of student achievement growth, classroom practices, and 
principal leadership. 


Key Questions Addressed: 


e What is the impact of the performance evaluation system on student achievement? 
e What is the impact on teachers’ classroom practices? 


e What are districts’ and educators’ experiences with implementation? 


Design: Within each of eight districts, approximately 15 elementary and middle schools were 
randomly assigned to receive the study’s measures of student achievement growth, 
classroom practices, and principal leadership during 2012-13 and 2013-14 (treatment 
group) or to participate only in the district's usual performance evaluation system during the 
same time period. In treatment schools, each year, teachers received four rounds of 
classroom observations and feedback sessions and information about their students’ 
achievement growth in math and/or reading using a statistical model known as value-added. 
Principals in treatment schools received feedback on their leadership practices twice per 
year. Study data collection included surveys of teachers and principals, observations of 
teachers’ classroom practices, and collection of student records data. The study also looks 
at the extent to which the measures were implemented as intended and whether the ratings 
from the measures reliably distinguish educator performance. 


Estimated or Actual Completion Date: A report on first year implementation findings was 
released in November 2016. A final report on impacts on educator practices and student 
achievement as well as implementation during the study’s second year is expected in fall 
2017. 


Key Findings: The main findings from the study’s first report are: 


e The study’s performance measures were implemented generally as planned. Teachers 
and principals received multiple rounds of ratings and feedback on their practices. 
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However, fewer principals and teachers accessed their student growth reports than the 
study intended. 


e Both classroom observation and student growth measures differentiated teacher 
performance, although observation scores were mostly at the upper end of the scale. 
Overall, observation scores varied across teachers, and both value-added scores and 
average classroom observation scores over the year had sufficient reliability to capture 
performance differences among some teachers. 


e The principal leadership measure differentiated performance, but principal self-ratings, 
teachers’ ratings of the principal, and the principal’s supervisor's ratings of the principal 
often differed. 


e Both teachers and principals in treatment schools reported receiving more feedback on 
their performance than did their counterparts. For example, teachers and principals in 
treatment schools reported spending more total time in performance feedback sessions 
across the year than teachers and principals in the control schools. 


6. Links to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/tq_performance.asp 


Impact Evaluation of the Teacher Incentive Fund 


1. Study Purpose: The TIF was renamed the Teacher and School Leader Incentive Program 
with reauthorization of the ESEA in the ESSA. The purpose of the TIF program is to develop 
and implement performance-based compensation systems (PBCSs) for teachers, principals, 
and other personnel in high-need schools. Research indicates that high-quality teachers are 
critical to raising student achievement in low-performing schools, but schools most in need 
often have difficulty in attracting and retaining high-quality teachers. Performance pay is a 
policy promoted by the TIF program to improve the quality of teachers in high-need schools. 
This evaluation studies performance pay that provides substantial and differentiated bonus 
pay to high-performing teachers in low-performing schools with high-need students. 


2. Key Questions Addressed: 


e What are the characteristics of all TIF grantee districts and their performance-based 
compensation systems? What implementation experiences and challenges did TIF 
districts encounter? 


e How do teachers and principals in schools that did or did not offer pay-for-performance 
bonuses compare on key dimensions, including their understanding of TIF program 
features, exposure to TIF activities, allocation of time, and attitudes toward teaching and 
the TIF program? 


e What is the impact of pay-for-performance bonuses on students’ achievement on state 
assessments of math and reading? 


e How do pay-for-performance bonuses affect educator mobility, including whether 
mobility differs by educator effectiveness? 


3. Design: Study schools were randomly assigned within a grant to either implement all 
components of the PBCS or the PBCS with a 1 percent across-the-board bonus in place of 
the differentiated effectiveness incentive component of the PBCS. Data collection will 
include a grantee survey, a survey of teachers and principals, teacher and principal school 
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assignment records, student record information (such as student demographics and student 
test scores), and grantee interviews to document implementation information, as well as to 
conduct impact analyses. 


Estimated or Actual Completion Date: The first report was released in September 2014. The 
second report was released in September 2015. The third report was released in August 
2016. The fourth and final report, which will include updated impacts, is expected in fall 
2017. 


Key Findings from the Third Report: The main findings among all TIF districts with 2010 
awards are: 


e Similar to the previous two years, most districts (88 percent) implemented at least three 
of the four required program components for teachers. 


e By the third year, reported implementation challenges decreased with no more than 
one-fifth of TIF districts reporting any major challenges. 


e For the subset of 10 districts that agreed to participate in a random assignment study, 
key findings on the effect of pay-for-performance on educators include the following: 


e After three years of TIF implementation, average student achievement remained 1 to 
2 percentile points higher in schools that offered pay-for-performance bonuses than in 
schools that did not. This difference was equivalent to a gain of about four additional 
weeks of learning. 


e Atleast half of the evaluation districts each year met the grant guidance for awarding 
differentiated performance bonuses for teachers. However, in each year, no more than 
2 of the 10 districts awarded bonuses for teachers that were substantial or challenging to 
earn. 


e Teachers’ understanding of performance measures continued to improve, but only about 
60 percent of teachers correctly reported being eligible for a performance bonus. In 
addition, teachers believed that the maximum bonus they could earn was no more than 
two-fifths the size of the actual maximum bonus that districts awarded, a finding similar 
to previous years. 


6. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tgq_incentive.asp 


Impact Evaluation of Support for Principals 


Ti 


Study Purpose: Title Il, Part A, the Improving Teacher State Formula Grants program, is the 
primary federal funding under ESEA to support high-quality educators. The program targets 
high-poverty districts and funds a broad array of allowable activities for principals and 
teachers, such as support for certification, teacher mentoring and induction, intensive 
professional development, recruitment, retention, and merit-based teacher and principal pay 
strategies as well as class size reduction. Principals, through a collective focus on 
instructional and organizational leadership and human capital management, have the 
potential to greatly influence the quality of instruction. However, there is limited evidence 
about the effectiveness of principal professional development programs and their ability to 
improve principals’ leadership skills and school quality. This evaluation studies professional 
development for principals and thus provides an important source of information for this 
program. 
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2. Key Questions Addressed: 


e What are the professional development experiences of principals? 


e What are the initial impacts on school climate and educator behaviors of providing 
principals structured and intensive professional development? 


e What are the impacts on teacher retention, the effectiveness of instructional staff, and 
student achievement of providing principals with structured and intensive professional 
development? 


3. Design: The study team will randomly assign within 10 districts a total of 100 elementary 


schools to a treatment or control group. Treatment group principals will be offered intensive 
professional development provided by the University of Washington’s Center for Educational 
Leadership during the 2015-16 and 2016-17 school years. The Center for Educational 
Leadership was competitively selected to provide the professional development focused on 
in this study. The professional development includes a heavy emphasis on instructional 
leadership activities, including support in conducting school walkthroughs and classroom 
observations with constructive feedback to facilitate teacher growth focused on improving 
student achievement. Control group principals will receive supports normally offered by the 
district. Data collection will include: information about the professional development 
delivered and experienced by the participating principals; teacher and principal surveys and 
periodic logs of principal daily activities to document intermediate outcomes, such as 
principal behaviors and school climate; and administrative records to document student 
outcomes (e.g., achievement, behavior, attendance) and teacher outcomes (e.g., retention 
of effective teachers, quality of newly hired teachers). 


Estimated or Actual Completion Date: The first report, which will focus on implementation 
and intermediate outcomes, is expected in spring 2018. 


5. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_principals.asp 


Study of Teacher Preparation Experiences and Early Teacher Effectiveness 


1. 


Study Purpose: A primary focus of the ESSA’s Title Il Part A is on the improvement of 
teacher quality. Little research exists, however, to inform how best to prepare teachers for 
the classroom. This study provides descriptive information on the preparation experiences of 
a large sample of novice teachers. It also will examine whether the instructional skills that 
teachers learn about and have opportunities to practice in their preparation programs are 
associated with teachers’ effectiveness once they are in the classroom. 


Key Questions Addressed: 


e To what extent do teachers report participating in various preparation experiences? How 
much variation is there within and between programs? 


e Among novice teachers, what are the relationships between the teachers’ experience in 
their preparation programs and the achievement of students in their classroom? 


Design: Approximately 3,200 novice language arts and/or math teachers from grades 4 
through 6 participated in the study. In spring 2015, the teachers responded to a survey 
focused on their preparation experiences related to instructional skills that have been shown 
to be associated with teacher value-added scores. For each instructional skill, the teachers 
were asked about the nature of their learning experiences, including how they learned (e.g., 
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through coursework or through practice in K-12 classroom) and the extent to which they 
found the experience useful. The study will also compute value-added scores for teachers, 
based on students’ state math and English language arts tests, and examine the 
relationships between teacher preparation experiences and teacher value-added scores. 


4. Estimated or Actual Completion Date: A report describing teachers’ preparation experiences 
and the relationship between experiences and teacher value-added scores is expected by 
fall 2017. 


5. Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/tq_teacherprep early.asp. 


Study of the Distribution of Effective Teaching 


1. Study Purpose: There is a persistent achievement gap in the United States where students 
from high-income families outperform those from low-income families on achievement tests. 
There is also substantial variation in the effectiveness of teachers. A key question for 
policymakers is whether policy initiatives focused on providing low-income students with 
equal access to effective teachers can address the achievement gap. This study provided 
information about the extent to which disadvantaged students received less-effective 
teaching than other students. The study also examined teacher mobility in participating 
districts and how patterns of mobility might contribute to unequal access. 


2. Key Questions Addressed: 


e Are low-income students taught by less effective teachers than high-income students? If 
so, to what extent would providing equal access to effective teachers reduce the student 
achievement gap? 


e Are there differences between high- and low-poverty schools in teacher hiring, transfer, 
and attrition? If so, are they consistent with inequitable access to effective teachers for 
low-income students? 


3. Design: The study documented low-income students’ access to effective teachers, as 
measured by value added across the 2008-09 through 2012-13 school years. The study 
also described district polices designed to address inequitable distribution of effective 
teaching implemented during those years. Lastly, the study examined teacher mobility 
patterns within participating districts. Annual data collection included district administrative 
records, such as student achievement to conduct value added analyses, as well as annual 
semi-structured interviews with district leadership to provide information on district policies. 
District personnel data were also collected to examine teacher mobility within participating 
districts. The study was conducted in 29 geographically dispersed school districts. 


4. Estimated or Actual Completion Date: The final report on school years 2008-09 through 
2012-13 was released in October 2016. 


5. Key Findings from the Final Report: The final report focused on low-income students’ access 
to effective teachers and teacher mobility patterns in 26 districts and found that: 


e There are small differences in the effectiveness of teachers of high- and low-income 


students, on average. The average teacher of a low-income student is just below the 
50th percentile of effectiveness based on value-added, while the average teacher of a 
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high-income student is at the 51st percentile. Providing low-income students with equally 
effective teachers would not substantively reduce the achievement gap. 


e Ina subset of the study districts, there is meaningful inequity in teacher effectiveness in 
math. In 3 of the 26 study districts, providing low-income students with teachers whose 
effectiveness is equal to that of high-income students over a five-year period would 
reduce the math achievement gap by at least a tenth of a standard deviation of student 
achievement, the equivalent of about 4 percentile points. 


e Teacher hiring patterns are consistent with small inequities in access to effective 
teachers. High-poverty schools have more newly hired teachers than low-poverty 
schools, but this difference is likely to have a small influence on equity because 
(1) relatively few teachers are new hires (11 percent of teachers in high-poverty schools 
and 5 percent in low-poverty schools), and (2) performance of newly hired teachers 
improves quickly. On average, newly hired teachers become as effective as the average 
teacher after one year. 


e Teacher transfer patterns are also consistent with small inequities in access to effective 
teachers. Teachers who transfer to schools in a higher poverty category are less 
effective (43rd percentile) than the average district teacher. Teachers who transfer to 
schools in a lower poverty category are nearly as effective (48th percentile) as the 
average district teacher. These patterns likely have a small influence on equity since just 
under 4 percent of all teachers transfer across poverty categories each year. 


e Teacher attrition patterns do not contribute to inequity. Teachers who leave a district are 
less effective (44th percentile) than the average teacher, and more teachers leave high- 
poverty schools than low-poverty schools (10 percent versus 7 percent, respectively). 


6. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_distribution.asp 
Other 


Evaluation of Investing in Innovation 


dz 


Study Purpose: |Investing in Innovation (i3) is a competitive grant program that funds entities 
with a record of improving student achievement in order to expand the implementation of, 
and investment in, evidence-based interventions to significantly improve student education 
outcomes. The i3 program supports three types of grants that are differentiated by level of 
prior evidence supporting the intervention, the scope of the proposed implementation, and 
the amount of funding to be provided. Scale-up and Validation grants require prior evidence 
of effectiveness, while Development grants support innovative strategies. 


Grantees receiving funds under this program are required to conduct an independent 
evaluation of their project and must agree, along with their independent evaluator, to 
cooperate with evaluation technical assistance provided by the Department and its 
contractor. The purpose of this technical assistance is to maximize the strength of the 
impact studies and the quality of their implementation data and performance feedback. 


2. Key Questions Addressed: 


e What types of supports were provided to i3 local evaluators and how were these 
supports implemented? 


e Were the i3-funded evaluations well-designed and well-implemented? 
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e What did the i3-funded evaluations add to the evidence base on educational 
interventions? 


3. Design: This study is descriptive and includes technical assistance to support the i3 local 
evaluators. The study also includes a systematic review of the completed evaluations 
against the WWC evidence standards as well as other criteria for assessing the strength of 
the impact and implementation aspects of the evaluations. 


4. Estimated or Actual Completion Date: The first report, which will focus primarily on findings 
for the first cohort (FY 2010), is expected in fall 2017. 


5. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/assistance_ita.asp 


Evaluation of NAEP Achievement Levels 


1. Study Purpose: Under the provisions of P.L. 107-279, the Secretary is required to provide 
for continuing review of the National Assessment of Educational Progress (NAEP) 
assessment. The legislation identifies the issues to be addressed in the reviews, one of 
which includes the requirement to evaluate whether the NAEP achievement levels, 
established by the National Assessment Governing Board, are “reasonable, valid, reliable 
and informative to the public.” Section 303(e)(2)(C) of the Education Sciences Reform Act of 
2002 (P.L. 107-279) states that NAEP achievement levels shall be used on a trial basis until 
the Commissioner of the National Center for Education Statistics (NCES) determines, as a 
result of an evaluation under Sec. 303(f), that such levels are “reasonable, valid, and 
informative to the public.” This independent evaluation was intended to provide IES and the 
Commissioner of NCES with information necessary to inform the decision about whether the 
current trial status of the NAEP achievement levels can be removed or whether they should 
remain in trial status. 


2. Key Objectives: 


e Determine how “reasonable, valid, reliable and informative to the public” will be 
operationalized in this study. 


e Identify the kinds of objective data and research findings that will be examined. 
e Review and analyze extant information related to the study’s purpose. 


e Gather other objective information from relevant experts and stakeholders, without 
creating burden for the public through new, large-scale data collection. 


e Organize, summarize, and present the findings from the evaluation in a written report, 
including a summary that is accessible for nontechnical audiences, discussing the 
strengths/weaknesses and gaps in knowledge in relation to the evaluation criteria. 


e Provide, prior to release of the study report, for an independent external review of that 
report for comprehensiveness, objectivity, and freedom from bias. 


3. Design: This study focused on the achievement levels used in reporting NAEP results for 
the reading and mathematics assessments in grades 4, 8, and 12. Specifically, the study 
reviewed developments over the past decade in the ways achievement levels for NAEP are 
set and used and will evaluate whether the resulting achievement levels are “reasonable, 
valid, reliable, and informative to the public.” The study relied on an independent committee 
of experts with a broad range of expertise related to assessment, statistics, social science, 
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and education policy. The project is receiving oversight from the Board on Testing and 
Assessment and the Committee on National Statistics of the National Research Council. 


Actual Completion Date: The prepublication version of the report from this study was 
released in November 2016 and may be downloaded at 
https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levels-for-mathematics- 
and-reading-on-the-national-assessment-of-educational-progress. 


5. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other_naep.asp 


Evaluation of the Comprehensive Technical Assistance Centers, FY 2012 Grantees 


Ts 


Study Purpose: The Comprehensive Technical Assistance Centers is a federally funded 
program currently authorized under the Educational Technical Assistance Act of 2002. The 
Department awarded five-year grants in FY 2012 to 22 Comprehensive Technical 
Assistance Centers with the purpose to help state education agencies build their capacity to 
implement state-level initiatives and to support district- and school-level initiatives that 
improve educational outcomes for all students, close achievement gaps, and improve the 
quality of instruction. The FY 2014 appropriation for the Centers was $48.4 million. This 
study will inform the Department of Education, the Comprehensive Center program, and the 
larger field about the design, implementation, and outcomes of the Centers’ work. 


Key Questions Addressed: 


e Program Design: How did the Centers design their work? How did Centers define 
capacity building? Did their definitions change over time? If so, how? What theories of 
action did Centers use to guide their general capacity-building work? Did the theories 
change over time? If so, how? How did Centers assess the needs of their 
constituencies? 


e Program Implementation: How did the Centers operate? What strategies did Centers 
employ to achieve their outcomes? To what extent did Centers implement technical 
assistance to their constituencies as planned? To what extent and how did Centers 
collaborate with each other? 


e Program Outcomes: What was the result of the Centers’ work? To what extent did 
Centers achieve their goals and objectives? 


Design: This evaluation is a multiyear descriptive study examining the Centers’ programs. 
Data on the Centers’ activities and outcomes will be collected during the FY 2015, FY 2016, 
and FY 2017 program years. The evaluation will describe how the individual Centers intend 
to build SEA capacity (their theories of action) and document what types of activities they 
actually conduct to build capacity. The evaluation plans to focus on Center projects in two 
priority areas: great teachers and leaders and early learning. Data collection will include: 

(1) the Centers’ management plans and technical assistance activity data; (2) interviews 
with staff from each Center; (3) interviews with technical assistance recipients; (4) a survey 
of Center staff; and (5) a survey of technical assistance recipients. Additionally, observations 
of technical assistance events will be conducted to inform project profiles that illustrate the 
strategies that the Centers use to support capacity building and achieve planned outcomes. 
This approach will yield a diverse set of data that can be analyzed and summarized using 
qualitative research methods and simple quantitative tabulations. 


4. Estimated or Actual Completion Date: A report describing findings is expected in 2018. 
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5. Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/other_techcenters12.asp 


Implementation and Impact Evaluation of Race to the Top: Final Report 


1. Study Purpose: RTT is a competitive grant program of the Department that funds states and 
districts planning to implement comprehensive education reform in one or more core areas. 
Since its first awards in 2010, RTT has funded general state competitions, state 
competitions focused on early learning, and district competitions focused on personalized 
learning. With funds from the Recovery Act, the general state competition awarded 
approximately $4 billion to states in support of comprehensive K-12 education reform in 
several core areas, including teachers and leaders, standards and assessments, data 
systems, and school turnaround. This study examined the implementation of RTT and its 
relationship to student outcomes, focusing on the initial general state competition for RTT 
that began in 2010. 


2. Key Questions Addressed: 


e Which policies and practices promoted by the RTT program do RTT states report using, 
and how do they compare to the policies and practices that non-RTT states report 
using? 


e ls receipt of an RTT grant related to improvement in student outcomes? 


3. Design: The RTT sample included all 50 states and DC. Data from interviews with all states 
and DC informed the first evaluation question. The second evaluation question was 
addressed using a short interrupted time series design with state-level NAEP data 
comparing, before and after the RTT competition, states that were awarded an RTT grant to 
states that applied for but were not awarded an RTT grant. 


4. Estimated or Actual Completion Date: The final report for RTT was released in October 
2016. 


5. Key Findings from the Final Report for RTT: 


e In four of six areas examined, 2010 RTT grantees reported using more policies and 
practices promoted by RTT than states that did not receive a grant: The four areas with 
differences were standards and assessments, teachers and leaders, school turnaround, 
and charter schools. The other two areas were state capacity and data systems. 


e 2011 RTT grantees reported using more policies and practices promoted by RTT than 
states that did not receive a grant in one area, which was teachers and leaders. 


e Across all states, use of RTT-promoted policies and practices were highest in the data 
systems area and lowest in the teachers and leaders area: States reported using 
76 percent of the 8 RTT-promoted practices examined in data systems, but only 
26 percent of the 39 practices in teachers and leaders. 


e The relationship between RTT and student outcomes was not clear: Trends in student 
outcomes could be interpreted as providing evidence that RTT had a positive effect, a 
negative effect, or no effect. 


6. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other_racetotop.asp 
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Implementation and Impact Evaluation of SIGs: Final Report 


1. Study Purpose: The SIG program is authorized through Title | of the ESEA and provides 
three-year awards to support turnaround in the nation’s persistently lowest-achieving 
schools. In FY 2009, the $546 million SIG appropriation was supplemented by $3 billion 
through the Recovery Act, for a total of $3.5 billion. SIG funds are distributed to states by 
formula based on Title | allocations. States then competitively award funds to districts 
applying on behalf of their eligible schools. Schools receiving SIG must implement one of 
four prescriptive intervention models: turnaround, transformation, closure, or restart. This 
study examined the implementation and impacts of SIG, focusing on the first cohort of SIG 
schools implementing intervention models beginning in the 2010-11 school year. 


2. Key Questions Addressed: 


e Are SlG-funded schools using the improvement or turnaround strategies promoted by 
the four SIG intervention models, and how do they compare to strategies in schools not 
implementing a SIG-funded intervention model? How are states and districts supporting 
such efforts? 


e Does receipt of SIG funding to implement a school intervention model have an impact on 
outcomes for low-performing schools? 


e ls implementation of the four school intervention models related to improvement in 
outcomes for low-performing schools? 


3. Design: The SIG sample included about 500 schools in 60 districts from 22 states. This 
sample was purposively selected to support a regression discontinuity design to address the 
fourth evaluation question, exploiting cutoff rules that states used to identify their persistently 
lowest-achieving schools as eligible for SIG to implement one of the four intervention 
models. Data from state and district interviews, as well as school surveys from the SIG 
sample, were used to inform the third and fifth evaluation questions. Student- and school- 
level achievement data were also collected from administrative records up to the 2012-13 
school year to inform the fourth and fifth evaluation questions. 


4. Estimated or Actual Completion Date: The final report for SIG was released in January 
2017. 


5. Key Findings from the Final Report for SIG: 


e SIG schools implementing one of the four models (transformation, turnaround, restart, or 
closure) reported using more practices than other schools: SIG schools reported using 
an average of 23 out of 35 practices, whereas other schools reported using 20 practices. 


e Across all schools, use of SIG-promoted practices was highest in the area of 
comprehensive instructional reform strategies and lowest in the area of operational 
flexibility and support: Schools reported using 89 percent of the eight SIG-promoted 
practices examined in the comprehensive instructional reform strategies area, but only 
43 percent of the two practices in operational flexibility and support (the other two areas 
examined were increasing teacher and principal effectiveness, and increasing learning 
time and creating community-oriented schools). 


e Implementing any of the four SIG models had no significant impacts on math or reading 
test scores, high school graduation, or college enrollment. 
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e Inelementary grades, student achievement gains did not differ across the four SIG 
models. In secondary grades, the turnaround model was associated with larger 
achievement gains than the transformation model. 


6. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other_racetotop.asp 


Implementation of Title I/II Program Initiatives 


1. Study Purpose: The Title | and Title II programs are part of the ESEA and are intended to 
help provide all students with equal access to education by providing financial assistance to 
schools and districts that have a high percentage of students from low-income families 
(Title |) and improving teacher and principal quality (Title Il). The previous reauthorization of 
ESEA, the No Child Left Behind Act of 2001 (NCLB), increased accountability through the 
use of assessments for students, requirements for all teachers to be highly qualified, and 
funding for supports and interventions for schools not achieving adequate yearly progress 
for the entire school population as well as for various subgroups of students. Subsequently, 
a majority of states received ESEA Flexibility beginning in 2012, which allowed particular 
NCLB requirements to be waived in exchange for a commitment to implement various 
reform principles. ESEA was reauthorized as the ESSA in December 2015, and it offers 
states and districts more autonomy than under NCLB and ESEA Flexibility, specifically in the 
areas of accountability, educator evaluation systems, and teacher qualification 
requirements. 


This study is designed to provide relevant data from states, districts, schools, and teachers 
on the implementation of programs and policies related to Title | and Title Il under NCLB and 
ESEA Flexibility (during the 2013-14 school year). It will also provide data from states and 
districts as they undergo the early stages of implementing the ESSA (during the 2017-18 
school year). 


2. Key Questions Addressed: 


e What content standards and high school graduation requirements are states adopting, 
and what materials and resources do states, districts, and schools provide to help 
teachers implement the state content standards? 


e What assessments do states and districts use (in terms of assessment format and 
coverage of grade levels and content areas), and what materials and resources do 
states, districts, and schools provide to support the implementation of assessments and 
use assessment data? 


e How has student achievement changed over time? 


e What elements are included in states’ accountability systems? How do states and 
districts identify and reward their highest-performing schools, how do they identify and 
support their lowest-performing schools, and how do they offer differentiated support for 
those schools that are neither highest-performing nor lowest-performing? 


e How do states and districts evaluate teacher and principal effectiveness and assess 
equitable distribution of teachers and principals, and what supports do states, districts, 
and schools provide to improve teacher and principal effectiveness? 


3. Design: The study is descriptive and nationally representative. Data were collected from all 
50 states and the District of Columbia, a nationally representative sample of districts and 
schools, and teachers within those schools through surveys in the 2013-14 school year. 
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Subsequently, data will be collected from all 50 states and the District of Columbia as well 
as the same nationally representative sample of districts through surveys in the 201 7-18 
school year. 


4. Estimated or Actual Completion Date: The first report, based on the findings from the 
2013-14 data collection, was released in January 2017. 


5. Key Findings from the First Report: 


Most states adopted and most principals and teachers reported implementing state 
standards that focused on college- and career-readiness. All but one state had 
committed to having college- and career-ready standards in place by 2013-14. A 
majority of principals (67-69 percent, depending on subject) reported fully implementing 
state content standards, and most teachers reported receiving professional development 
relevant to state content standards (79 percent of teachers) and weekly use of aligned 
instructional activities (92 percent of teachers). 


Many state assessments incorporated more sophisticated response formats to better 
assess students’ college- and career-readiness. In their reading/English language arts 
(ELA) summative assessments, many states (24-36, depending on grade level) reported 
using extended constructed-response formats, a type of response format intended to 
assess higher-order thinking skills. Nineteen states used this response format in math 
assessments. 


States used ESEA flexibility to reset their accountability goals and to target a narrower 
set of schools for additional support. Forty-three states had received ESEA Flexibility for 
the 2013-14 school year. The most common accountability goal adopted by states with 
ESEA Flexibility (28 of the 43 states) was reducing by half the percentage of students 
and subgroups not proficient in 6-8 years. States with ESEA Flexibility identified 

5 percent of Title | schools as lowest performing and an additional 10 percent of Title | 
schools with substantial student achievement gaps, compared to non-Flexibility states 
that reported identifying 43 percent of Title | schools as lowest performing. 


Almost all states adopted new laws or regulations related to educator evaluation 
systems between 2009 and 2014, and most districts reported full or partial 
implementation in 2013-14. Only four states had not adopted new teacher evaluation 
laws or regulations by 2014, and a majority (59 percent) of districts reported fully 
implementing, piloting, or partially implementing a new teacher evaluation system. 
However, few districts (18 percent) reported using evaluation system measures of 
student achievement growth and classroom practice consistent with emerging research. 


Proficiency rates on the National Assessment of Educational Progress (NAEP) slightly 
increased from 2005 to 2015, with the largest increases in 4th and 8th grades and 
smaller or no increases in 12th grade. Overall proficiency rates increased by statistically 
significant levels of 4—5 percentage points in 4th and 8th grade reading and math and by 
2 percentage points in 12th grade reading. Statistically significant increases in 
proficiency were also evident for economically disadvantaged students in both subjects 
and across all three grades (by 4—7 percentage points), and in the large majority of 
individual states (46—51 states, depending on grade and subject). 


6. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other _titlel.asp 
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Progress and Challenges in Developing Tiered Quality Rating and Improvement Systems 
(TQRIS) in the Round 1 Race to the Top-Early Learning Challenge (ELC) States 


1. Study Purpose: ELC aims to improve early learning and development programs so children 
enter kindergarten ready to succeed. One of the key areas of reform for the program 
focuses on the design and implementation of Tiered Quality Rating and Improvement 
Systems (TQRIS) that can provide parents and stakeholders with information on the quality 
of early learning programs. Since ELC grantee states are in the process of developing, 
refining, and/or implementing their TQRIS, this evaluation will describe the current state 
plans and implementation status. 


2. Key Question Addressed: 


e How are TQRIS ratings defined, collected, and generated in the nine Round 1 ELC 
states and how might the structure and implementation of the rating system influence the 
reliability and validity of the ratings? 


3. Design: This is a descriptive study that involved collecting and analyzing various data from 
the nine Round 1 ELC grantee states. The study collected and conducted a targeted review 
of documents describing the structure of TQRIS, including component measures and the 
quality indicators used to evaluate preschool programs, how these are combined to 
generate overall ratings, the availability of TQRIS and child outcome data, and policies 
regarding the administration of kindergarten entry assessments (KEAs) in each Round 1 
state. The study also included interviews with state administrators to confirm and clarify the 
information obtained from documents and, as needed, to gather information that could not 
be obtained from the document reviews. 


4. Estimated or Actual Completion Date: An evaluation brief is scheduled for completion in 
summer 2017. 


5. Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other_rtt.asp 


Policy and Program Studies Service (PPSS) 
TEACH Grant Study 


1. Study Purpose: The TEACH Grant program provides grants up to $4,000 a year to students 
who are completing coursework needed to begin teaching. If a recipient does not complete 
four years of service in a high-need subject at a high-need school within eight years after 
completing their coursework, their grant funds are converted to a direct unsubsidized loan. 
This study examines how institutions support and inform students who are eligible for 
TEACH Grants. In addition, the study examines why some participants do not meet TEACH 
Grant service requirements. Lastly, the study examines factors associated with grant 
recipients meeting and not meeting service requirements. 


2. Key Questions Addressed: 


e How are TEACH Grants administered in institutions of higher education? 
e Why do some TEACH Grant recipients fail to meet program service requirements? 


e What are the factors associated with TEACH Grant recipients meeting and not meeting 
the grant service requirements? 
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3. Design: The study includes surveys of 1) 479 institutions that administered at least 


10 TEACH Grants in the 2014-15 award year and 2) a sample of 500 TEACH Grant 
recipients. In addition, the study includes case studies of six institutions and an analysis of 
administrative data. 


4. Estimated Completion Date: The final report is scheduled for completion in summer 2017. 


National Survey on High School Strategies Designed to Help At-Risk Students Graduate 


Ts 


Study Purpose: This nationally representative survey of high school administrators is 
examining strategies that high schools use to reduce students’ likelihood of dropping out of 
high school and to increase their likelinood of attaining a high school credential. The survey 
seeks information on the prevalence of high school graduation strategies, the students who 
participate in them, and how high schools deliver services or interventions as part of that 
strategy. The Department will release a set of issue briefs based on the survey data to 
describe the prevalence and characteristics of dropout prevention strategies for at-risk youth 
and will compare high schools with high and low graduation rates, among other school 
characteristics. 


Key Questions Addressed: 


e What strategies are high schools implementing to help students stay in school and 
graduate? Do these strategies vary for high schools with high or low graduation rates? 


e How many students are served through each of these strategies? Are the strategies 
focused on particular student populations? 


e How do schools deliver services or interventions for each of the strategies? What 
specific services are provided, and who provides the services? 


Design: The study is conducting a web-based survey of a nationally representative sample 
of approximately 2,000 high school administrators. 


Estimated Completion Date: |ssue briefs are scheduled for completion between winter 
2016-17 and spring 2017. 


Implementation Study of the Turnaround School Leaders Program 


1. 


Study Purpose: This study will examine the implementation of the Turnaround School 
Leaders Program (TSLP). This study seeks to generate information to help policymakers 
and practitioners who struggle with the challenges of developing leaders to turn around low- 
performing schools and to add to the field’s general body of knowledge about developing 
turnaround leadership pipelines. 


2. Key Questions Addressed: 


e How do TSLP projects identify, develop, and support leaders for low-performing 
schools? 


e What role do project partners play in implementing projects and helping grantees to 
achieve project goals? 


e How have grantees modified projects to adapt to challenges or meet the demands of 
changing circumstances? 


FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan—U.S. Department of Education 177 


APPENDICES 
SUMMARY OF PERFORMANCE EVALUATIONS 


e How are grantees measuring the success of their TSLP projects, and do early outcome 
data show promising results? 


e How have the turnaround school leaders grants contributed to developing a sustainable, 
long-term pipeline of leaders for turnaround schools? 


3. Design: The study will include surveys of 12 TSLP cohort 1 grantees and case studies of 


seven TSLP cohort 1 grantees, including each grantees’ partners; an analysis of extant 
data, including grantee applications, early outcomes data, and other relevant project-specific 
data. 


4. Estimated Completion Date: The final report is scheduled for completion in fall 2017. 


Study of School Climate Transformation Grants 


Ts 


Study Purpose: The study will describe how states and school districts that participate in the 
School Climate Transformation Grant (SCTG) program are coordinating services and 
supports with certain other related federal programs administered by the Departments of 
Health and Human Services and Justice. 


Key Questions Addressed: 


e What does coordination between SCTG and Project AWARE grantees look like? 

e What does coordination between SCTG and DOJ grantees look like? 

e What did grantees say about the value of coordination? 

e What were the challenges and lessons learned? 

Design: The study will conduct telephone interviews with representatives from 38 state and 


local grantees to explore the ways in which grantees coordinate services, the benefits that 
grantees have experienced from program coordination, and challenges and lessons learned. 


4. Estimated Completion Date: The report is scheduled for completion in late fall/winter 2017. 


Study of Title | Schoolwide and Targeted Assistance Programs 


1. 


Study Purpose: The study will examine how Title | schoolwide programs use the schoolwide 
flexibility to design services and strategies to address the needs of low-achieving students 
and subgroups, and how such strategies compare to approaches used in targeted 
assistance programs. The study will include interviews and analysis of extant data in 
approximately 30 Title | schools, including both schoolwide and targeted assistance 
programs, as well as surveys of principals and district administrators for a representative 
sample of Title | schools. 


2. Key Questions Addressed: 


e How do schoolwide and targeted assistance programs use Title | funds to improve 
student achievement, particularly for low-achieving subgroups? 


e How do districts and schools make decisions about how to use Title | funds in 
schoolwide programs and targeted assistance programs? 


e To what extent do schoolwide programs commingle Title | funds with other funds or 
coordinate the use of Title | funds with other funds? 
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3. Design: The study will include both in-depth case studies and surveys of a nationally 


representative sample of 350 districts and 1,400 schools. The case studies will include 
approximately 35 Title | schools including both schoolwide and targeted assistance 
programs. Data collection for the case studies will include site visits and interviews with two 
to four staff members in each school and approximately two staff members in each district, 
as well as extant documents and data, including Title | budgets and plans. 


4. Estimated Completion Date: The report is scheduled for completion in spring 2018. 


Study of Digital Learning Resources for Instructing English Learners 


Te 


Study Purpose: This study will examine the use of digital learning resources (DLRs) to 
support the English language acquisition and academic achievement of English learners in 
K-12 education. The study will explore the range of DLRs that are available for use with 
English learner students, examine how districts and schools select and use these apps, and 
consult an expert panel of technology developers, practitioners, and education researchers 
for ideas on ways to improve the design and use of apps to support learning for English 
learner students. The study will culminate in a final report that presents findings from the 
study, as well as two short field-focused toolkits or guides for educators and technology 
developers that present key information from the study in a manner that will be accessible 
and useful for those audiences. 


Key Questions Addressed: 


e How do districts and teachers identify and select DLRs in general? How do districts and 
teachers identify and select DLRs specifically to support English learner students? 


e What are the types and characteristics of DLRs that districts most commonly report as 
used to support English learners? What are the types and characteristics of DLRs that 
teachers most commonly report that they use in instructing and structuring learning 
activities for their English learner students? 


e How do teachers of English learner students use DLRs in the instruction of English 
learner students? 


e To what extent do teachers receive professional development or other supports for 
effective use of DLRs for instruction? Which professional development approaches do 
teachers report to be most helpful in supporting their use of DLRs in instruction? 


e What are barriers to and supports for (1) the use of DLRs in instruction of English learner 
students and (2) the use of DLRs by students at home? How can districts, schools, and 
DLR developers address these? 


e How do districts and teachers define and measure the success of their use of technology 
to support English learner students? 


e How could developers and practitioners improve the usefulness of DLRs for instructing 
English learner students? 


Design: The study will explore the range of such apps that are available for use with English 
learner students, examine how districts and schools select and use these apps, and consult 
an expert panel of technology developers, practitioners, and education researchers for ideas 
on ways to improve the design and use of apps to support learning for English learner 
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students. It will rely on six key components: (1) a literature review; (2) market research on 
existing DLRs for K-12 instruction; (3) survey of school districts; (4) survey of teachers; 
(5) case studies; and (6) an expert panel of technology developers, practitioners, and 
education researchers. 


4. Estimated Completion Date: The report is scheduled for completion in spring 2018. 


Study of the Title Ill Native American and Alaska Native Children in Schools Program 


1. 


Study Purpose: The study will examine services and strategies for English learners in the 
Native American, Alaska Native, Native Hawaiian, and Pacific Islander communities that are 
supported through the Native American and Alaska Native Children in Schools (NAM) 
program. The study includes case study of 25 NAM grantees, and examines: (1) the types of 
services and strategies that NAM funds support to address the instructional needs of these 
communities and develop student proficiency in both English and (optionally) native 
languages; (2) how grantees plan, implement, and evaluate their respective projects; 

(3) how grantees coordinate and prioritize the use of NAM funds in relation to other federal, 
state, and local resources; and (4) how grantees use data and evidence to inform program 
implementation and meet U.S. Department of Education reporting requirements. 


Key Questions Addressed: 


e How does the NAM program support grantees in providing services to Native American, 
Alaska Native, Native Hawaiian, and Pacific Islander children, especially through teacher 
training and professional development, curriculum development, parent engagement, 
and English (especially academic English) and native language instruction? How do 
NAM grantees incorporate technology to support and/or preserve heritage languages 
through dual-language or other forms of English language instruction? How many 
students are served through each of these strategies? Are the strategies focused on 
particular student populations? 


e What challenges do NAM grantees face in providing funded services, and what steps 
have grantees taken to overcome these challenges? To what extent does the 
Department or other external entities provide support to overcome these challenges? 
What lessons have the grantees learned? 


e How do schools deliver services or interventions for each of the strategies? What 
specific services are provided, and who provides the services? 


e What are the roles and responsibilities among (as applicable) tribal entities, public 
schools, local education districts, and state agencies in implementing NAM grants and 
meeting federal reporting requirements? 


e What are NAM grant stakeholders’ perceptions of community participation and student 
engagement in language instruction and other educational programs? 


Design: The study will include site visits to NAM grantees to obtain detailed information 
regarding how they are implementing the NAM program and meeting the needs of their 
students and communities, as well as telephone surveys or interviews of local grant 
coordinators and state directors of Indian education to inform the case study site visits. The 
study will also utilize extant data, including grant applications. 


4. Estimated Completion Date: The report is scheduled for completion in spring 2018. 
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Evaluation of the Title |, Part D Neglected or Delinquent Program 


1s 


Study Purpose: The study will examine the implementation of educational programs for 
children and youth in residential facilities and correctional institutions funded under Title |, 
Part D of ESEA. The study will be informed by surveys of state grantees and local 
subgrantees to examine the types of services and strategies that Part D funds support, how 
state and local agencies assist students in transitioning back to schools, how state 
correctional facilities implement institution-wide Part D projects, and how grantees assess 
the educational outcomes of participating students. 


Key Questions Addressed: 


e What types of services and strategies do Title |, Part D funds support for children and 
youth in correctional and child welfare settings? 


e How do correctional facilities and child welfare agencies assist students in transitioning 
back to districts and schools, including those outside their jurisdictions? 


e How do state correctional facilities plan and implement institution-wide Part D projects? 


e How do grantees assess the educational outcomes of students participating in 
Part D-funded educational programs? 


Design: The study will include a review of extant data, a review of literature related to 
programs for neglected and delinquent youth, surveys of state and local coordinators of 
Title |-Part D funded programs, and site visits to state agencies, school districts, correctional 
institutions, and child welfare facilities to obtain more detailed information on how grantees 
and subgrantees are implementing the Part D programs and how they are meeting the 
needs of their students. 


4. Estimated Completion Date: The report is scheduled for completion in spring 2018. 


Evaluation of the Migrant Education Program 


1. 


Study Purpose: This study will examine how states, districts, and schools are providing 
instructional supports and assessing highly mobile migratory students, as well as examine 
state plans for implementing the new accountability requirements (as they pertain to 
migratory students) under the ESSA. The Migrant Education Program (MEP) is a state 
program, giving states flexibility in how they allocate funds to serve migratory students, 
allowing the local entities that serve migratory students to be both local educational 
agencies (LEAs) and local operating agencies (LOAs). 


2. Key Questions Addressed: 


e How do state and local grantees respond to federal requirements for serving migratory 
children? 


e What services are provided to migratory students? 


e How do MEP grantees collaborate with other programs and agencies to address the 
needs of highly mobile students? 


e How do migrant programs support students in earning high school diplomas and 
equivalency, and preparing for postsecondary education and the workforce? 
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3. Design: The study will include survey of states, a nationally representative sample of district 


programs serving migrant children, site visits to state and local grantees. The contractor 
shall prepare a final report that integrates findings from the surveys and case studies as well 
as from an extant data analysis. 


4. Estimated Completion Date: The report is scheduled for completion in fall 2018. 


Evaluation of the Indian Education LEA Grants Program 


iF 


Study Purpose: This study will examine the implementation of the Indian Education Local 
Education Agency (LEA) Grants Program funded under Title VI of the ESEA, as amended 
by the ESSA. The study will document the scope of activities funded by the Indian Education 
LEA Grants Program and will examine the LEA and tribal-level implementation of the grants. 
Specifically, the study will examine the processes used to identify and count eligible children 
and how grantees establish LEA program priorities and implement grant-funded services. 


Key Questions Addressed: 


e What strategies do grantees use to identify and count program-eligible children? 
e How do grantees plan services? 
e What services do Indian Education LEA Grants fund? 


e How do grantees assess the project outcomes of students participating in Indian 
Education LEA Grants Program-funded educational programs? 


Design: This study will consist of four key components: (1) analysis of extant data including 
APRs and Electronic Application System for Indian Education (EASIE) data; (2) review of 
relevant literature; (3) survey of 1,300 coordinators of Indian Education LEA grant programs; 
and (4) case studies of nine districts. 


4. Estimated Completion Date: The report is scheduled for completion in fall 2018. 


Study of Weighted Student Funding Systems 


1s 


Study Purpose: This study is examining weighted student funding (WSF) and school-based 
budgeting (SBB) systems, which are methods for providing funds to schools based on the 
numbers and, in the case of WSF, types of students they served. The study will examine 
how districts have implemented SBB and WSF systems for allocating funds to schools and 
how these districts and their schools compare with districts using traditional systems for 
allocating school resources. 


2. Key Questions Addressed: 


e How are resources allocated to schools in districts with SBB or WSF systems compared 
with districts with more traditional resource allocation practices? 


e In what ways do schools have autonomy and control over resource allocation decisions, 
and how does this vary between districts with SBB or WSF and other districts? 


e How has the implementation of WSF systems affected the distribution of funding 
provided to schools? 


e What challenges did districts and schools experience in implementing SBB and WSF, 
and how did they respond to those challenges? 
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3. Design: To obtain detailed information about the implementation, benefits, and challenges of 
WSF systems, this study will conduct case studies in nine districts using such systems, 
including site visits that include in-person interviews and collection and analysis of extant 
documents and data such as school budgets and planning documents. In addition, the study 
will administer surveys to a nationally representative survey of districts and schools to 
enable the study to ground the case study data in the larger context of WSF, SBB, and 
traditional methods of resource allocation. 


4. Estimated Completion Date: The report is scheduled for completion in winter 2018-19. 
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Appendix E: Glossary of Acronyms and Abbreviations 


2FA 
AARTS 
AFR 

AIR 

AP 

APG 
APP 
APR 

AY 

CAP Goals 
CCSSO 
CEDS 
CEELO 
CFR 
COP 
CoSN 
CoSTEM 
CPS 
CRDC 
CTE 

CY 
DATA Act 
DEC 
DHS 
DLR 

DM 

DOJ 


Two-Factor Authentication 

Audit Accountability and Resolution Tracking System 
Agency Financial Report 

American Institutes for Research 

Advanced Placement 

Agency Priority Goal 

Annual Performance Plan 

Annual Performance Report 

Academic Year 

Cross-Agency Priority Goals 

Council of Chief State School Officers 
Common Education Data Standards 

Center for Enhancing Early Learning Outcomes 
Code of Federal Regulations 

Community of Practice 

Consortium for School Networking 

Committee on STEM Education 

Current Population Survey 

Civil Rights Data Collection 

Career and Technical Education 

Calendar Year 

Digital Accountability and Transparency Act of 2014 
Division of Early Childhood 

U.S. Department of Homeland Security 

Digital Learning Resources 

Document Management 


U.S. Department of Justice 
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DOL 
DRT 
DST 
EAC 
EAG 
EASN 
ECR 
ECS 
EDGAR 
EDSOC 
ELC 
ELC TA 
EMAPS 
EPG 
EQUIP 
ERIC 
ESEA 
ESSA 
EVS 
FAFSA 
FERPA 
FEVS 
FFEL 
FFY 
FPCO 
FSA 
FY 
GANs 
GEAR Up 
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U.S. Department of Labor 

Data Request Team 

Data Strategy Team 

Equity Assistance Center 

Enhanced Assessment Grants 

Equitable Access Support Network 

Early Complaint Resolution 

Education Commission on the States 

Education Department General Administrative Regulations 
Education Department Security Operations Center 
Early Learning Challenge 

Early Learning Challenge Technical Assistance Program 
EDFacts Metadata and Process System 

Evidence Planning Group 

Educational Quality through Innovative Partnerships 
Education Resources Information Center 
Elementary and Secondary Education Act of 1965 
Every Student Succeeds Act 

Employee Viewpoint Survey 

Free Application for Federal Student Aid 

Family Educational Rights and Privacy Act 

Federal Employee Viewpoint Survey 
Federal Family Education Loan 

Federal Fiscal Year 

Family Policy Compliance Office 

Federal Student Aid 

Fiscal Year 

Grant Award Notices 


Gaining Early Awareness and Readiness for Undergraduate Programs 
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GPRAMA 
GSA 
HBCUs 
HEA 
HEAL 
HHS 

HR 
HSI-STEM 
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Government Performance and Results Act of 1993 
GPRA Modernization Act of 2010 

General Services Administration 

Historically Black Colleges and Universities 

Higher Education Act of 1965 

Health Education Assistance Loans 

U.S. Department of Health and Human Services 
Human Resources 

Hispanic Serving Institution STEM Articulation Program 
U.S. Department of Housing and Urban Development 
Investing in Innovation Program 

Individuals with Disabilities Education Act 
Institute of Education Sciences 
Institutions of Higher Education 

Interagency Policy Board 

Integrated Postsecondary Education Data System 
Information Technology 

Job Opportunity Announcement 

Kindergarten Entry Assessment 

Lyndon B. Johnson 

Local Educational Agency 

Literacy Information and Communication System 
Local Operating Agency 

Migrant Education Program 

Magnet School Assistance Program 

Mathematics and Science Partnerships 

Not Applicable 

National Assessment of Educational Progress 


Native American and Alaska Native Children in Schools Program 
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NASA 
NCER 
NCES 
NIA 
NIEER 
NRC 
NSF 
OCFO 
OCIO 
OCR 
OCTAE 
OEL 
OELA 
OESE 
OET 
OGC 
OHR 
OIG 
Oll 
OM 
OMB 
OPE 
OPEPD 
OPM 
OSEP 
OSERS 
OSP 
OSS 
P3 
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National Aeronautics and Space Administration 
National Center for Education Research 
National Center for Educational Statistics 

Notice Inviting Applications 

National Institute for Early Education Research 
National Research Council 

National Science Foundation 

Office of the Chief Financial Officer 

Office of the Chief Information Officer 

Office for Civil Rights 

Office of Career, Technical, and Adult Education 
Office of Early Learning 

Office of English Language Acquisition 

Office of Elementary and Secondary Education 
Office of Educational Technology 

Office of the General Counsel 

Office of Human Resources 

Office of Inspector General 

Office of Innovation and Improvement 

Office of Management 

Office of Management and Budget 

Office of Postsecondary Education 

Office of Planning, Evaluation, and Policy Development 
Office of Personnel Management 

Office of Special Education Programs 

Office of Special Education and Rehabilitative Services 
Opportunity Scholarship Program 

Office of State Support 


Performance Partnership Pilots for Disconnected Youth 
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PAYE 
PBCS 
PCP 
PDG 
PDG TA 
PFS 
POC 
POR 
PPIRS 
PPSS 
PSLF 
R3 
RCE 
RDA 
REAP 
REPAYE 
RLIS 
RMS 
Rtl 
RTL 
RTT 
SASS 
SBB 
SCTG 
SEA 
SFA 
SIG 
SIP 
SLDS 
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Pay As You Earn 

Performance-Based Compensation System 
Potomac Center Plaza 

Preschool Development Grants 

Preschool Development Grant Technical Assistance 
Pay For Success 

Principal Operating Component 

Program of Requirements 

Past Performance Information Retrieval System 
Policy and Program Studies Service 

Public Service Loan Forgiveness 

Rapid Rent Reduction Initiative 

Rapid Cycle Evaluation 

Results Driven Accountability 

Rural Education Achievement Program 
Revised Pay As You Earn 

Rural and Low-Income Schools Program 

Risk Management Service 

Response to Intervention 

Ready to Learn 

Race to the Top 

Schools and Staffing Survey 

School-Based Budgeting 

School Climate Transformation Grant Program 
State Educational Agency 

Student Financial Assistance 

School Improvement Grant 

Strengthening Institutions Program 


Statewide Longitudinal Data System 
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SOAR Act 
SPPAD 
SRSA 
SSIP 
STEM 
SWD 

SY 

TBD 
TEACH 
TIF 
TQRIS 
Treasury 
TRIO 
TSLP 
U.S. 

VR 
WHI-HBCU 
WIOA 
WSF 
WWC 
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Scholarships and Opportunities for Results Act 

Student Privacy Policy and Assistance Division 

Small, Rural School Achievement Program 

State Systemic Improvement Plan 

Science, Technology, Engineering, and Mathematics 

Students with Disabilities 

School Year 

To Be Determined 

Teacher Education Assistance for College and Higher Education Grant 
Teacher Incentive Fund 

Tiered Quality Rating and Improvement System 

U.S. Department of Treasury 

Federal TRIO Programs 

Turnaround School Leaders Program 

United States 

Vocation Rehabilitation 

White House Initiative on Historically Black Colleges and Universities 
Workforce Innovation and Opportunity Act 

Weighted Student Funding 

What Works Clearinghouse 
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OUR MISSION IS TO PROMOTE STUDENT ACHIEVEMENT AND PREPARATION FOR 
GLOBAL COMPETITIVENESS BY FOSTERING EDUCATIONAL EXCELLENCE AND 
ENSURING EQUAL ACCESS. 
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